dev@glassfish.java.net

Re: deadlock in LogManager

From: Lloyd Chambers <Lloyd.Chambers_at_Sun.COM>
Date: Fri, 24 Apr 2009 13:50:06 -0700

I've been seeing this deadlock for some weeks now, it's not new, I
just didn't realize it for what it was. Here is another example:


=============================
"pool-3-thread-1":
   waiting to lock monitor 0x0000000139afcc28 (object
0x00000001159a5988, a java.util.logging.LogManager),
   which is held by "GlassFish Kernel Main Thread"
"GlassFish Kernel Main Thread":
   waiting to lock monitor 0x0000000139afa810 (object
0x000000012b1c1140, a java.lang.Class),
   which is held by "pool-3-thread-1"

------------------------------------------


Apr 24, 2009 1:48:26 PM
com.sun.enterprise.admin.launcher.GFLauncherLogger info
INFO: JVM invocation command line:
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java
-cp
/work/v3/glassfishv3/glassfish/modules/glassfish.jar
-XX:+UnlockDiagnosticVMOptions
-XX:MaxPermSize=192m
-XX:NewRatio=2
-XX:+LogVMOutput
-XX:LogFile=/work/v3/glassfishv3/glassfish/domains/domain1/logs/jvm.log
-Xmx512m
-client
-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver
-Djavax.net.ssl.trustStore=/work/v3/glassfishv3/glassfish/domains/
domain1/config/cacerts.jks
-Djavax.net.ssl.keyStore=/work/v3/glassfishv3/glassfish/domains/
domain1/config/keystore.jks
-Djava.security.policy=/work/v3/glassfishv3/glassfish/domains/domain1/
config/server.policy
-Dcom.sun.aas.instanceRoot=/work/v3/glassfishv3/glassfish/domains/
domain1
-
Dcom
.sun
.enterprise
.config
.config_environment_factory_class
=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory
-Djava.security.auth.login.config=/work/v3/glassfishv3/glassfish/
domains/domain1/config/login.conf
-Djava.endorsed.dirs=/work/v3/glassfishv3/glassfish/lib/endorsed
-Dcom.sun.aas.installRoot=/work/v3/glassfishv3/glassfish
-Djava.ext.dirs=/System/Library/Frameworks/JavaVM.framework/Versions/
1.6.0/Home/lib/ext:/System/Library/Frameworks/JavaVM.framework/
Versions/1.6.0/Home/jre/lib/ext:/work/v3/glassfishv3/glassfish/domains/
domain1/lib/ext:/work/v3/glassfishv3/javadb/lib
com.sun.enterprise.glassfish.bootstrap.ASMain
-domainname
domain1
-asadmin-args
start-domain,,,--verbose
-instancename
server
-watchdog
true
-verbose
true
-debug
false
-asadmin-classpath
/v3/glassfishv3/bin/../glassfish/modules/admin-cli.jar
-asadmin-classname
com.sun.enterprise.admin.cli.AsadminMain
-domaindir
/work/v3/glassfishv3/glassfish/domains/domain1
Watchdog is running. The Domain is remote-restart enabled.
Apr 24, 2009 1:48:27 PM
com.sun.enterprise.admin.launcher.GFLauncherLogger info
INFO: Successfully launched in 17 msec.
Apr 24, 2009 1:48:27 PM com.sun.enterprise.glassfish.bootstrap.ASMain
main
INFO: Launching GlassFish on Apache Felix OSGi platform
Apr 24, 2009 1:48:27 PM
com.sun.enterprise.glassfish.bootstrap.AbstractMain flushAndCreate
INFO: Removing cache dir /work/v3/glassfishv3/glassfish/domains/
domain1/felix-cache/gf left from a previous run

Welcome to Felix.
=================

felix.fileinstall.poll (ms) 5000
felix.fileinstall.dir /work/v3/glassfishv3/glassfish/modules
felix.fileinstall.debug 1
felix.fileinstall.bundles.new.start false
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [103]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.core.kernel [164]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.util [48]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle com.sun.enterprise.config [53]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.config-api [52]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.branding [37]
Apr 24, 2009 1:48:29 PM AppServerStartup run
INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
Apr 24, 2009 1:48:29 PM
com.sun.enterprise.v3.services.impl.LogManagerService postConstruct
INFO: Starting GlassFish Application Server.
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.internal-api [127]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.registration.glassfish-registration
[106]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.jms.core [148]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.connectors.runtime [57]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-naming [105]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jta [162]
Apr 24, 2009 1:48:29 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-api [91]
^\2009-04-24 13:48:34
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
mixed mode):

"pool-3-thread-1" daemon prio=5 tid=0x0000000103188000 nid=0x13a237000
waiting for monitor entry [0x000000013a235000..0x000000013a236ad0]
    java.lang.Thread.State: BLOCKED (on object monitor)
2009-04-24 13:48:34
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
mixed mode):

"domain1-StderrDrainer" daemon prio=5 tid=0x00000001030ce000
nid=0x116c5a000 runnable [0x0000000116c59000..0x0000000116c59ad0]
    java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:199)
        at java.lang.UNIXProcess
$DeferredCloseInputStream.read(UNIXProcess.java:227)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        - locked <0x000000010b110108> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read(FilterInputStream.java:90)
        at
com
.sun
.enterprise
.universal
.process
.ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:69)
        at java.lang.Thread.run(Thread.java:637)

"domain1-StdoutDrainer" daemon prio=5 tid=0x00000001030cd800
nid=0x116b57000 runnable [0x0000000116b56000..0x0000000116b56ad0]
    java.lang.Thread.State: RUNNABLE
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:260)
        at java.io.BufferedOutputStream.write(BufferedOutputStream.java:105)
        - locked <0x000000010cb05968> (a java.io.BufferedOutputStream)
        at java.io.PrintStream.write(PrintStream.java:430)
        - locked <0x000000010cafe5f8> (a java.io.PrintStream)
        at
com
.sun
.enterprise
.universal
.process
.ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:72)
        at java.lang.Thread.run(Thread.java:637)

"process reaper" daemon prio=5 tid=0x00000001030cb000 nid=0x116980000
runnable [0x000000011697f000..0x000000011697fad0]
    java.lang.Thread.State: RUNNABLE
        at java.lang.UNIXProcess.waitForProcessExit(Native Method)
        at java.lang.UNIXProcess.access$900(UNIXProcess.java:17)
        at java.lang.UNIXProcess$2$1.run(UNIXProcess.java:86)

"Low Memory Detector" daemon prio=5 tid=0x0000000103044800
nid=0x116105000 runnable [0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=0x0000000103043000 nid=0x116002000
waiting on condition [0x0000000000000000..0x00000001160003e0]
    java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=0x0000000103041800 nid=0x115eff000
waiting on condition [0x0000000000000000..0x0000000115efd3e0]
    java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
nid=0x115dfc000 waiting on condition
[0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x115ce2000 in
Object.wait() [0x0000000115ce1000..0x0000000115ce1ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010cafa630> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x000000010cafa630> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000103028800
nid=0x115bd9000 in Object.wait()
[0x0000000115bd8000..0x0000000115bd8ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010cafa710> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x000000010cafa710> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x0000000103001000 nid=0x103901000 in Object.wait()
[0x0000000103900000..0x0000000103900c70]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010b1100b0> (a java.lang.UNIXProcess)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.UNIXProcess.waitFor(UNIXProcess.java:115)
        - locked <0x000000010b1100b0> (a java.lang.UNIXProcess)
        at com.sun.enterprise.admin.launcher.GFLauncher.wait(GFLauncher.java:
298)
        at
com
.sun
.enterprise.admin.launcher.GFLauncher.launchInstance(GFLauncher.java:
248)
        at
com
.sun
.enterprise
.admin.launcher.GFDomainLauncher.internalLaunch(GFDomainLauncher.java:
47)
        at
com.sun.enterprise.admin.launcher.GFLauncher.launch(GFLauncher.java:68)
        - locked <0x000000010cb49490> (a
com.sun.enterprise.admin.launcher.GFDomainLauncher)
        at
com
.sun
.enterprise
.admin
.cli.StartDomainCommand.runCommandNotEmbedded(StartDomainCommand.java:
99)
        at
com
.sun
.enterprise
.admin.cli.StartDomainCommand.runCommand(StartDomainCommand.java:46)
        at
com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:160)
        at com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:150)
        at com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:76)

"VM Thread" prio=9 tid=0x0000000103020800 nid=0x115ad6000 runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000103046000
nid=0x116208000 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x0000000103002000
nid=0x103a04000 runnable
        at java.util.logging.LogManager.addLogger(LogManager.java:326)
        - waiting to lock <0x00000001159a5988> (a java.util.logging.LogManager)
        at java.util.logging.Logger.getLogger(Logger.java:321)
        - locked <0x000000012b1c1140> (a java.lang.Class for
java.util.logging.Logger)
        at com.sun.pkg.client.Image.<clinit>(Image.java:762)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.getUpdateCenterImage(RegistrationUtil.java:
175)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.setUpdateCenterUUID(RegistrationUtil.java:
182)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.synchUUID(RegistrationUtil.java:169)
        at
com
.sun
.enterprise
.registration.glassfish.PingService.postConstruct(PingService.java:85)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:170)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b4801e0> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at com.sun.enterprise.v3.server.AppServerStartup
$2.run(AppServerStartup.java:200)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"Thread-1" prio=5 tid=0x0000000103184800 nid=0x13a134000 waiting on
condition [0x000000013a133000..0JNI global references: 657
x000000013a133ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x000000010b4803e0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at
com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:
498)
        at com.sun.enterprise.server.logging.GFFileHandler
$3.run(GFFileHandler.java:246)

"pool-2-thread-1" daemon prio=5 tid=0x0000000103180000 nid=0x13a031000
waiting on condition [0x000000013a030000..0x000000013a030ad0]
    java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x000000010b480600> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:
198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
        at
java
.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:906)
        at java.lang.Thread.run(Thread.java:637)

"GlassFish Kernel Main Thread" prio=5 tid=0x0000000103173800
nid=0x139f2e000 waiting for monitor entry
[0x0000000139f27000..0x0000000139f2dad0]
    java.lang.Thread.State: BLOCKED (on object monitor)
        at java.util.logging.Logger.getLogger(Logger.java:274)
        - waiting to lock <0x000000012b1c1140> (a java.lang.Class for
java.util.logging.Logger)
        at java.util.logging.LogManager.addLogge
r(LogManager.java:399)
        - locked <0x00000001159a5988> (a java.util.logging.LogManager)
        at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
        at
com
.sun
.enterprise
.resource.rm.ResourceManagerImpl.<clinit>(ResourceManagerImpl.java:61)
        at
com
.sun
.enterprise.resource.pool.PoolManagerImpl.<init>(PoolManagerImpl.java:
98)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:
72)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:71)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b1e6d18> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat$1.get(Habitat.java:252)
        at java.util.AbstractList$Itr.next(AbstractList.java:345)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:171)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:121)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - Heap
locked <0x000000010b1b0760> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010afd9430> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b182820> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat$1.get(Hab def new generation
total 2304K, used 375K [0x000000010aed0000, 0x000000010b160000,
0x000000010cad0000)
   edenitat.java:252)
        at java.util.AbstractList$Itr.next(AbstractList.java:345)
        at
com
.sun
.enterprise
.naming
.impl
.SerialInitContextFactory
.getInitialContext(SerialInitContextFactory.java:112)
        at
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
        at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:
288)
        at javax.naming.InitialContext.init(InitialContext.java:223)
        at javax.naming.InitialContext.<init>(InitialContext.java:175)
        at
com
.sun
.enterprise
.naming
.impl
.GlassfishNamingManagerImpl.<init>(GlassfishNamingManagerImpl.java:94)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:
72)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:71)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010afd9430> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b4844d0> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b484678> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at
com
.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:
215)
        at com.sun.enterprise.v3.server.AppServerStartup
$1.run(AppServerStartup.java:127)

"pool-1-thread-4" prio=5 tid=0x0000000103153800 nid=0x139d53000
waiting on condition [0x0000000139d52000..0x0000000139d52ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116619f18> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transaction space
1984K, 6% used [0x000000010aed0000, 0x000000010aef09e8,
0x000000010b0c0000)
   froms.java:142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"pool-1-thread-3" prio=5 tid=0x0000000103152800 nid=0x139c50000
waiting on condition [0x0000000139c4f000..0x0000000139c4fad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x000000011661a170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"pool-1-thread-2" prio=5 tid=0x000000010314c000 nid=0x139ac2000
waiting on condition [0x0000000139ac1000..0x0000000139ac1ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x000000011661a3c8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"pool-1-thread-1" prio=5 tid=0x000000010314a800 nid=0x1399bf000
waiting on condition [0x00000001399be000..0x00000001399bead0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.p space 320K, 76% used [0x000000010b110000,
0x000000010b14d600, 0x000000010b160000)
ark(Native Method)
        - parking to wait for <0x000000011661a620> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"{felix.fileinstall.poll=5000,
felix.fileinstall.bundles.new.start=false, felix.fileinstall.dir=/work/
v3/glassfishv3/glassfish/modules/, felix.fileinstall.debug=1}" daemon
prio=5 tid=0x00000001030fd800 nid=0x13924d000 waiting on condition
[0x000000013924c000..0x000000013924cad0]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        a to space 320K, 0% used [0x000000010b0c0000,
0x000000010b0c0000, 0x000000010b110000)
  tenured generation total 5312K, used 734K [0x000000010cad0000,
0x000000010d000000, 0x00000001102d0000)
    the space 5312K, 13% usedt
org
.apache.felix.fileinstall.DirectoryWatcher.run(DirectoryWatcher.java:
122)

"Configuration Updater" daemon prio=5 tid=0x00000001030f4000
nid=0x13914a000 in Object.wait()
[0x0000000139149000..0x0000000139149ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001161236c8> (a java.util.LinkedList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
        - locked <0x00000001161236c8> (a java.util.LinkedList)

"telnetconsole.Listener" daemon prio=5 tid=0x00000001030e6800
nid=0x138f32000 runnable [0x0000000138f31000..0x0000000138f31ad0]
    java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        - locked <0x00000001161237b0> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.felix.shell.remote.Listener$Acceptor.run(Listener.java:
137)
        at java.lang.Thread.run(Thread.java:637)

"FelixPackageAdmin" daemon prio=5 tid=0x00000001030a8800
nid=0x1389c4000 in Object.wait()
[0x00000001389c3000..0x00000001389c3ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115a210f0> (a
org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Object.wait(Object.java:485)
        at
org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:
314)
        - locked <0x0000000115a210f0> (a
org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Thread.run(Thread.java:637)

"FelixStartLevel" daemon prio=5 tid=0x00000001030a7800 nid=0x13872c000
in Object.wait() [0x000000013872b000..0x000000013872bad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115a21aa8> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:
241)
        - locked <0x0000000115a21aa8> (a java.util.ArrayList)
        at java.lang.Thread.run(Thread.java:637)

"FelixDispatchQueue" daemon prio=5 tid=0x000000010309c800
nid=0x1388c1000 in Object.wait()
[0x00000001388c0000..0x00000001388c0ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115a21b78> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at
org
.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:
875)
        - locked <0x0000000115a21b78> (a java.util.ArrayList)
        at org.apache.felix.framework.util.EventDispatcher.access
$000(EventDispatcher.java:50)
        at org.apache.felix.framework.util.EventDispatcher
$1.run(EventDispatcher.java:102)
        at java.lang.Thread.run(Thread.java:637)

"OSGi Framework Launcher" daemon prio=5 tid=0x0000000103082800
nid=0x1380be000 in Object.wait()
[0x00000001380bc000..0x00000001380bdad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115a24e58> (a
org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:48)
        - locked <0x0000000115a24e58> (a
org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.Felix.waitForStop(Felix.java:785)
        at org.apache.felix.main.Main.main(Main.java:215)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.sun.enterprise.glassfish.bootstrap.ASMainFelix
$1.run(ASMainFelix.java:91)
        at java.lang.Thread.run(Thread.java:637)

"Low Memory Detector" daemon prio=5 tid=0x0000000103044800
nid=0x137ad5000 runnable [0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=0x0000000103043000 nid=0x1379d2000
waiting on cond [0x000000010cad0000, 0x000000010cb87818,
0x000000010cb87a00, 0x000000010d000000)
  compacting perm gen total 21248K, used 8712K [0x00000001102d0000,
0x0000000111790000, 0x00000001156d0000)
    the space 21248K, 41% used [0x00000001102d0000,
0x0000000110b523d0, 0x0000000110b52400, 0x0000000111790000)
No shared spaces configured.

ition [0x0000000000000000..0x00000001379d03e0]
    java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=0x0000000103041800 nid=0x1378cf000
waiting on condition [0x0000000000000000..0x00000001378cd3e0]
    java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
nid=0x1377cc000 waiting on condition
[0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x1376b2000 in
Object.wait() [0x00000001376b1000..0x00000001376b1ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001159a2ea8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x00000001159a2ea8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000103028800
nid=0x1375a9000 in Object.wait()
[0x00000001375a8000..0x00000001375a8ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001159a2ab8> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x00000001159a2ab8> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x0000000103001000 nid=0x103901000 in Object.wait()
[0x0000000103900000..0x0000000103900c40]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001159e22e8> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1161)
        - locked <0x00000001159e22e8> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1214)
        at
com
.sun
.enterprise
.glassfish.bootstrap.ASMainFelix.launchOSGiFW(ASMainFelix.java:110)
        at
com.sun.enterprise.glassfish.bootstrap.ASMainOSGi.run(ASMainOSGi.java:
137)
        at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:65)

"VM Thread" prio=9 tid=0x0000000103020800 nid=0x1374a6000 runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000103046000
nid=0x137bd8000 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x0000000103002000
nid=0x103a04000 runnable
JNI global references: 762


Found one Java-level deadlock:
=============================
"pool-3-thread-1":
   waiting to lock monitor 0x0000000139afcc28 (object
0x00000001159a5988, a java.util.logging.LogManager),
   which is held by "GlassFish Kernel Main Thread"
"GlassFish Kernel Main Thread":
   waiting to lock monitor 0x0000000139afa810 (object
0x000000012b1c1140, a java.lang.Class),
   which is held by "pool-3-thread-1"

Java stack information for the threads listed above:
===================================================
"pool-3-thread-1":
        at java.util.logging.LogManager.addLogger(LogManager.java:326)
        - waiting to lock <0x00000001159a5988> (a java.util.logging.LogManager)
        at java.util.logging.Logger.getLogger(Logger.java:321)
        - locked <0x000000012b1c1140> (a java.lang.Class for
java.util.logging.Logger)
        at com.sun.pkg.client.Image.<clinit>(Image.java:762)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.getUpdateCenterImage(RegistrationUtil.java:
175)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.setUpdateCenterUUID(RegistrationUtil.java:
182)
        at
com
.sun
.enterprise
.registration
.glassfish.RegistrationUtil.synchUUID(RegistrationUtil.java:169)
        at
com
.sun
.enterprise
.registration.glassfish.PingService.postConstruct(PingService.java:85)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:170)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b4801e0> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at com.sun.enterprise.v3.server.AppServerStartup
$2.run(AppServerStartup.java:200)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)
"GlassFish Kernel Main Thread":
        at java.util.logging.Logger.getLogger(Logger.java:274)
        - waiting to lock <0x000000012b1c1140> (a java.lang.Class for
java.util.logging.Logger)
        at java.util.logging.LogManager.addLogger(LogManager.java:399)
        - locked <0x00000001159a5988> (a java.util.logging.LogManager)
        at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
        at
com
.sun
.enterprise
.resource.rm.ResourceManagerImpl.<clinit>(ResourceManagerImpl.java:61)
        at
com
.sun
.enterprise.resource.pool.PoolManagerImpl.<init>(PoolManagerImpl.java:
98)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:
72)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:71)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b1e6d18> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat$1.get(Habitat.java:252)
        at java.util.AbstractList$Itr.next(AbstractList.java:345)
        at java.util.AbstractCollection.toArray(AbstractCollection.java:171)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:121)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b1b0760> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010afd9430> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b182820> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat$1.get(Habitat.java:252)
        at java.util.AbstractList$Itr.next(AbstractList.java:345)
        at
com
.sun
.enterprise
.naming
.impl
.SerialInitContextFactory
.getInitialContext(SerialInitContextFactory.java:112)
        at
javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:667)
        at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:
288)
        at javax.naming.InitialContext.init(InitialContext.java:223)
        at javax.naming.InitialContext.<init>(InitialContext.java:175)
        at
com
.sun
.enterprise
.naming
.impl
.GlassfishNamingManagerImpl.<init>(GlassfishNamingManagerImpl.java:94)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:
72)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:71)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010afd9430> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b4844d0> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at org.jvnet.hk2.component.Habitat.getComponent(Habitat.java:335)
        at com.sun.hk2.component.AbstractWombImpl
$1.getValue(AbstractWombImpl.java:140)
        at
org.jvnet.hk2.component.InjectionManager.inject(InjectionManager.java:
81)
        at
com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:166)
        at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:90)
        at java.security.AccessController.doPrivileged(Native Method)
        at
com.sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:
87)
        at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:77)
        at
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
58)
        - locked <0x000000010b484678> (a
com.sun.hk2.component.SingletonInhabitant)
        at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
        at
com
.sun
.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:
60)
        at
com
.sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:
215)
        at com.sun.enterprise.v3.server.AppServerStartup
$1.run(AppServerStartup.java:127)

Found 1 deadlock.

Heap
  def new generation total 6720K, used 4984K [0x000000010aed0000,
0x000000010b640000, 0x0000000115970000)
   eden space 5824K, 71% used [0x000000010aed0000,
0x000000010b2e7140, 0x000000010b480000)
   from space 896K, 88% used [0x000000010b480000, 0x000000010b5471d8,
0x000000010b560000)
   to space 896K, 0% used [0x000000010b560000, 0x000000010b560000,
0x000000010b640000)
  tenured generation total 15180K, used 13830K [0x0000000115970000,
0x0000000116843000, 0x000000012aed0000)
    the space 15180K, 91% used [0x0000000115970000,
0x00000001166f19a8, 0x00000001166f1a00, 0x0000000116843000)
  compacting perm gen total 21248K, used 12166K [0x000000012aed0000,
0x000000012c390000, 0x0000000136ed0000)
    the space 21248K, 57% used [0x000000012aed0000,
0x000000012bab1880, 0x000000012bab1a00, 0x000000012c390000)
No shared spaces configured.





On Apr 24, 2009, at 12:20 PM, Lloyd Chambers wrote:

> While testing some code today, I came across this deadlock in
> LogManager.
>
> Is this a JDK bug, or a bug due to something we've inserted into the
> logging mechanism?
>
>
>
> ^CThe domain was stopped.
> llcMP:code llc$ r3
>
> Apr 24, 2009 12:14:51 PM
> com.sun.enterprise.admin.launcher.GFLauncherLogger info
> INFO: JVM invocation command line:
> /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/
> java
> -cp
> /work/v3/glassfishv3/glassfish/modules/glassfish.jar
> -XX:+UnlockDiagnosticVMOptions
> -XX:MaxPermSize=192m
> -XX:NewRatio=2
> -XX:+LogVMOutput
> -XX:LogFile=/work/v3/glassfishv3/glassfish/domains/domain1/logs/
> jvm.log
> -Xmx512m
> -client
> -Djdbc.drivers=org.apache.derby.jdbc.ClientDriver
> -Djavax.net.ssl.trustStore=/work/v3/glassfishv3/glassfish/domains/
> domain1/config/cacerts.jks
> -Djavax.net.ssl.keyStore=/work/v3/glassfishv3/glassfish/domains/
> domain1/config/keystore.jks
> -Djava.security.policy=/work/v3/glassfishv3/glassfish/domains/
> domain1/config/server.policy
> -Dcom.sun.aas.instanceRoot=/work/v3/glassfishv3/glassfish/domains/
> domain1
> -
> Dcom
> .sun
> .enterprise
> .config
> .config_environment_factory_class
> =
> com
> .sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory
> -Djava.security.auth.login.config=/work/v3/glassfishv3/glassfish/
> domains/domain1/config/login.conf
> -Djava.endorsed.dirs=/work/v3/glassfishv3/glassfish/lib/endorsed
> -Dcom.sun.aas.installRoot=/work/v3/glassfishv3/glassfish
> -Djava.ext.dirs=/System/Library/Frameworks/JavaVM.framework/Versions/
> 1.6.0/Home/lib/ext:/System/Library/Frameworks/JavaVM.framework/
> Versions/1.6.0/Home/jre/lib/ext:/work/v3/glassfishv3/glassfish/
> domains/domain1/lib/ext:/work/v3/glassfishv3/javadb/lib
> com.sun.enterprise.glassfish.bootstrap.ASMain
> -domainname
> domain1
> -asadmin-args
> start-domain,,,--verbose
> -instancename
> server
> -watchdog
> true
> -verbose
> true
> -debug
> false
> -asadmin-classpath
> /v3/glassfishv3/bin/../glassfish/modules/admin-cli.jar
> -asadmin-classname
> com.sun.enterprise.admin.cli.AsadminMain
> -domaindir
> /work/v3/glassfishv3/glassfish/domains/domain1
> Watchdog is running. The Domain is remote-restart enabled.
> Apr 24, 2009 12:14:51 PM
> com.sun.enterprise.admin.launcher.GFLauncherLogger info
> INFO: Successfully launched in 13 msec.
> Apr 24, 2009 12:14:51 PM
> com.sun.enterprise.glassfish.bootstrap.ASMain main
> INFO: Launching GlassFish on Apache Felix OSGi platform
>
> Welcome to Felix.
> =================
>
> felix.fileinstall.poll (ms) 5000
> felix.fileinstall.dir /work/v3/glassfishv3/glassfish/
> modules
> felix.fileinstall.debug 1
> felix.fileinstall.bundles.new.start false
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [103]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.core.kernel [164]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.util [48]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle com.sun.enterprise.config [53]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.admin.config-api [52]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.branding [37]
> Apr 24, 2009 12:14:52 PM AppServerStartup run
> INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
> Apr 24, 2009 12:14:52 PM
> com.sun.enterprise.v3.services.impl.LogManagerService postConstruct
> INFO: Starting GlassFish Application Server.
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.internal-api [127]
> Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.registration.glassfish-
> registration [106]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.jms.core [148]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.connectors.runtime [57]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.glassfish-naming [105]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.transaction.jta [162]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.glassfish-api [91]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.container-common [67]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.ejb.ejb-container [77]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.deployment.autodeploy [71]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.transaction.jts [163]
> Apr 24, 2009 12:14:53 PM
> com.sun.enterprise.transaction.JavaEETransactionManagerSimplified
> initDelegates
> INFO: Using
> com
> .sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate
> as the delegate
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.connectors.work-management [216]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.connectors.internal-api [56]
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.flashlight.framework [82]
> Apr 24, 2009 12:14:53 PM
> org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread startConnector
> INFO: Started JMXConnector, JMXService URL = service:jmx:rmi:///jndi/
> rmi://192.168.1.5:8686/jmxrmi
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle com.sun.grizzly.grizzly-config [111]
> Apr 24, 2009 12:14:53 PM
> com.sun.enterprise.v3.services.impl.GrizzlyProxy start
> INFO: Listening on port 8080
> Apr 24, 2009 12:14:53 PM
> com.sun.enterprise.v3.services.impl.GrizzlyService createNetworkProxy
> INFO: Network listener http-listener-2 on port 8181 disabled per
> domain.xml
> Apr 24, 2009 12:14:53 PM
> com.sun.enterprise.v3.services.impl.GrizzlyProxy start
> INFO: Listening on port 4848
> Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.deployment.common [73]
> Apr 24, 2009 12:14:53 PM
> com.sun.enterprise.v3.server.AppServerStartup run
> INFO: GlassFish v3 startup time : Felix(1253ms) startup
> services(693ms) total(1946ms)
> Apr 24, 2009 12:14:56 PM
> org.glassfish.admin.mbeanserver.ConnectorStartupService
> $BootAMXListener handleNotification
> INFO: ConnectorStartupService.BootAMXListener: connection made,
> booting AMX MBeans
> Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.amx-impl [19]
> Apr 24, 2009 12:14:56 PM
> INFO: org.glassfish.admin.amx.loader.AMXStartupService
> Apr 24, 2009 12:14:56 PM
> INFO: Got loader for AMXStartupServiceMBean:
> org.glassfish.admin.amx.loader.AMXStartupService_at_237dabb0
> Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.security.realms [187]
> Apr 24, 2009 12:14:56 PM
> org.glassfish.admin.amx.loader.AMXStartupService startAMX
> INFO: AMXStartupService: loaded 117 AMX MBeans in 315 ms (wall-clock
> time)
> Apr 24, 2009 12:14:56 PM
> org.glassfish.admin.mbeanserver.ConnectorStartupService
> $BootAMXListener handleNotification
> INFO: ConnectorStartupService.BootAMXListener: connection made,
> booting AMX MBeans
> Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.amx-core-impl [13]
> Apr 24, 2009 12:14:56 PM
> INFO: ProxyFactory: mDomainRootObjectName =
> v3:pp=,type=DomainRoot,name=v3
> Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.amx-j2ee-impl [21]
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/,type=SystemInfo
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/,type=J2EEDomain,j2eeType=J2EEDomain,name=v3
> Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.amx-config-impl [9]
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/
> J2EEDomain,type=J2EEServer,name=das,j2eeType=J2EEServer
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/J2EEDomain/
> J2EEServer[das],type=JVM,j2eeType=JVM,J2EEServer=das
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/,type=domain
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain,type=system-applications
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain,type=applications
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain,type=resources
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-
> resource,name=jdbc/__TimerPool
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-
> resource,name=jdbc/__default
> Apr 24, 2009 12:14:56 PM
> INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-connection-
> pool,name=__TimerPool
> Apr 24, 2009 12:14:57 PM
> org.glassfish.admin.amx.impl.config.ConfigBeanJMXSupport <init>
> WARNING: ConfigBeanJMXSupport (AMX): working around @Configured bug
> for com.sun.grizzly.config.dom.ThreadPool, using "ThreadPoolId" as
> the key attribute
> Apr 24, 2009 12:14:57 PM OSGiModuleImpl start
> INFO: Started bundle org.glassfish.common.amx-ext-impl [17]
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
> pool[__TimerPool],type=property,name=databaseName
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
> pool[__TimerPool],type=property,name=connectionAttributes
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-connection-
> pool,name=DerbyPool
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
> pool[DerbyPool],type=property,name=PortNumber
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
> pool[DerbyPool],type=property,name=Password
> Apr 24, 2009 12:14:57 PM
> INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
> pool[DerbyPool],type=property,name=User
>
>
>
> ^\2009-04-24 12:18:30
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
> mixed mode):
>
> "domain1-StderrDrainer" daemon prio=5 tid=0x00000001030cd800
> nid=0x116d76000 runnable [0x0000000116d75000..0x0000000116d75ad0]
> java.lang.Thread.State: RUNNABLE
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:199)
> at java.lang.UNIXProcess
> $DeferredCloseInputStream.read(UNIXProcess.java:227)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> - locked <0x000000010b113820> (a java.io.BufferedInputStream)
> at java.io.FilterInputStream.read(FilterInputStream.java:90)
> at
> com
> .sun
> .enterprise
> .universal
> .process
> .ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:69)
> at java.lang.Thread.run(Thread.java:637)
>
> "domain1-StdoutDrainer" daemon prio=5 tid=0x00000001030cd000
> nid=0x116c73000 runnable [0x0000000116c72000..0x0000000116c72ad0]
> java.lang.Thread.State: RUNNABLE
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:199)
> at java.lang.UNIXProcess
> $DeferredCloseInputStream.read(UNIXProcess.java:227)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> - locked <0x000000010b113a48> (a java.io.BufferedInputStream)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
> - locked <0x000000010b1138f8> (a java.io.BufferedInputStream)
> at java.io.FilterInputStream.read(FilterInputStream.java:90)
> at
> com
> .sun
> .enterprise
> .universal
> .process
> .ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:69)
> at java.lang.Thread.run(Thread.java:637)
>
> "process reaper" daemon prio=5 tid=0x00000001030cc000
> nid=0x116b70000 runnable [0x0000000116b6f000..0x0000000116b6fad0]
> java.lang.Thread.State: RUNNABLE
> at java.lang.UNIXProcess.waitForProcessExit(Native Method)
> at java.lang.UNIXProcess.access$900(UNIXProcess.java:17)
> at java.lang.UNIXProcess$2$1.run(UNIXProcess.java:86)
>
> "Low Memory Detector" daemon prio=5 tid=0x0000000103044800
> nid=0x116105000 runnable [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread1" daemon prio=9 tid=0x0000000103043000
> nid=0x116002000 waiting on condition
> [0x0000000000000000..0x00000001160003e0]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=9 tid=0x0000000103041800
> nid=0x115eff000 waiting on condition
> [0x0000000000000000..0x0000000115efd3e0]
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
> nid=0x115dfc000 waiting on condition
> [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x115ce2000 in
> Object.wait() [0x0000000115ce1000..0x0000000115ce1ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010cafb828> (a java.lang.ref.ReferenceQueue
> $Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x000000010cafb828> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> "Reference Handler" daemon prio=10 tid=0x0000000103028800
> nid=0x115bd9000 in Object.wait()
> [0x0000000115bd8000..0x0000000115bd8ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010cafb908> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x000000010cafb908> (a java.lang.ref.Reference$Lock)
>
> "main" prio=5 tid=0x0000000103001000 nid=0x103901000 in
> Object.wait() [0x0000000103900000..0x0000000103900c70]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010b110250> (a java.lang.UNIXProcess)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.UNIXProcess.waitFor(UNIXProcess.java:115)
> - locked <0x000000010b110250> (a java.lang.UNIXProcess)
> at
> com.sun.enterprise.admin.launcher.GFLauncher.wait(GFLauncher.java:298)
> at
> com
> .sun
> .enterprise.admin.launcher.GFLauncher.launchInstance(GFLauncher.java:
> 248)
> at
> com
> .sun
> .enterprise
> .admin
> .launcher.GFDomainLauncher.internalLaunch(GFDomainLauncher.java:47)
> at
> com.sun.enterprise.admin.launcher.GFLauncher.launch(GFLauncher.java:
> 68)
> - locked <0x000000010cb499e0> (a
> com.sun.enterprise.admin.launcher.GFDomainLauncher)
> at
> com
> .sun
> .enterprise
> .admin
> .cli
> .StartDomainCommand.runCommandNotEmbedded(StartDomainCommand.java:99)
> at
> com
> .sun
> .enterprise
> .admin.cli.StartDomainCommand.runCommand(StartDomainCommand.java:46)
> at
> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:
> 160)
> at com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:
> 150)
> at com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:76)
>
> "VM Thread" prio=9 tid=0x0000000103020800 nid=0x115ad6000 runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x0000000103046000
> nid=0x116208000 waiting on condition
>
> "Exception Catcher Thread" prio=10 tid=0x0000000103002000
> nid=0x103a04000 runnable
> 2009-04-24 12:18:30
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
> mixed mode):
>
> "AMXConfigLoader.AMXConfigLoaderThread" daemon prio=5
> tid=0x0000000103321000 nid=0x13bc2b000 waiting on condition
> [0x000000013bc2a000..0x000000013bc2aad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000116623cf0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> JNI global references: 657
>
> Heap
> def new generation total 2304K, used 298K [0x000000010aed0000,
> 0x000000010b160000, 0x000000010cad0000)
> eden space 1984K, 2% used at
> java
> .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:
> 358)
> at
> org
> .glassfish
> .admin
> .mbeanserver.PendingConfigBeansNew.take(PendingConfigBeansNew.java:
> 115)
> at org.glassfish.admin.amx.impl.config.AMXConfigLoader
> $AMXConfigLoaderThread.doRun(AMXConfigLoader.java:482)
> at org.glassfish.admin.amx.impl.config.AMXConfigLoader
> $AMXConfigLoaderThread.run(AMXConfigLoader.java:448)
>
> "ComplianceMonitor.ValidatorThread" daemon prio=5
> tid=0x0000000103317800 nid=0x13bb28000 waiting for monitor entry
> [0x000000013bb26000..0x000000013bb27ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000011599a0b8> (a
> java.util.logging.LogManager)
> at java.util.logging.Logger.getLogger(Logger.java:275)
> - locked <0x000000012b1c11b0> (a java.lang.Class for
> java.util.logging.Logger)
> at com.sun.jmx.trace.TraceManager.getLogger(TraceManager.java:166)
> at com.sun.jmx.trace.TraceManager.isSelected(TraceManager.java:197)
> at com.sun.jmx.trace.Trace.isSelected(Trace.java:84)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .isTraceOn(DefaultMBeanServerInterceptor.java:1832)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .getAttribute(DefaultMBeanServerInterceptor.java:657)
> at
> com
> .sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
> 638)
> at
> org
> .glassfish
> .admin
> .amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:
> 251)
> at
> org
> .glassfish
> .admin
> .amx
> .util
> .jmx.MBeanProxyHandler.getAttributeNoThrow(MBeanProxyHandler.java:263)
> at
> org
> .glassfish
> .admin
> .amx.core.proxy.AMXProxyHandler.getChildren(AMXProxyHandler.java:792)
> at
> org
> .glassfish
> .admin
> .amx.core.proxy.AMXProxyHandler.childrenSet(AMXProxyHandler.java:810)
> at
> org
> .glassfish
> .admin
> .amx
> .core.proxy.AMXProxyHandler.handleSpecialMethod(AMXProxyHandler.java:
> 421)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:
> 594)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
> at $Proxy66.childrenSet(Unknown Source)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 226)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 234)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 245)
> at
> org
> .glassfish
> .admin
> .amx.impl.path.PathnamesImpl.listObjectNames(PathnamesImpl.java:261)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
> 1082)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .invoke(DefaultMBeanServerInterceptor.java:836)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at
> javax
> .management
> .MBeanServerInvocationHandler
> .invoke(MBeanServerInvocationHandler.java:288)
> at
> org
> .glassfish
> .admin.amx.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:
> 467)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:
> 605)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
> at $Proxy69.listObjectNames(Unknown Source)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 318)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 331)
> at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
> $ValidatorThread.doRun(ComplianceMonitor.java:134)
> at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
> $ValidatorThread.run(ComplianceMonitor.java:118)
>
> "pool-1-thread-6" prio=5 tid=0x00000001032f3800 nid=0x13c168000
> waiting on condition [0x000000013c167000..0x000000013c167ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000116ac5278> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.con [0x000000010aed0000, 0x000000010aedb240,
> 0x000000010b0c0000)
> from space 320K, 79% used [0x000000010b110000, 0x000000010b14f690,
> 0x000000010b160000)
> to space 320K, 0% used [0x000000010b0c0000, 0x000000010b0c0000,
> 0x000000010b110000)
> tenured generation total 5312K, used 735K [0x000000010cad0000,
> 0x000000010d000000, 0x00000001102d0000)
> the space 5312K, 13% used [0x000000010cad0000,
> 0x000000010cb87fc0, 0x000000010cb88000, 0x000000010d000000)
> compacting perm gen total 21248K, used 8712K [0x00000001102d0000,
> 0x0000000111790000, 0x00000001156d0000)
> thecurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "AMXConfigLoader.AMXConfigLoaderThread" daemon prio=5
> tid=0x00000001032f3000 nid=0x13c065000 waiting on condition
> [0x000000013c064000..0x000000013c064ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000116641c68> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java
> .util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:
> 358)
> at
> org
> .glassfish
> .admin.mbeanserver.PendingConfigBeans.take(PendingConfigBeans.java:
> 117)
> at org.glassfish.admin.amx.config.AMXConfigLoader
> $AMXConfigLoaderThread.doRun(AMXConfigLoader.java:521)
> at org.glassfish.admin.amx.config.AMXConfigLoader
> $AMXConfigLoaderThread.run(AMXConfigLoader.java:488)
>
> "JMX server connection timeout 51" daemon prio=5
> tid=0x00000001032d4000 nid=0x13ba25000 in Object.wait()
> [0x000000013ba24000..0x000000013ba24ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000116ac55f0> (a [I)
> at java.lang.Object.wait(Object.java:485)
> at com.sun.jmx.remote.internal.ServerCommunicatorAdmin
> $Timeout.run(ServerCommunicatorAdmin.java:132)
> - locked <0x0000000116ac55f0> (a [I)
> at java.lang.Thread.run(Thread.java:637)
>
> "RMI TCP Connection(4)-192.168.1.5" daemon prio=5
> tid=0x00000001032d3800 nid=0x13b922000 waiting for monitor entry
> [0x000000013b91d000..0x000000013b921ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.Logger.getLogger(Logger.java:274)
> - waiting to lock <0x000000012b1c11b0> (a java.lang.Class for
> java.util.logging.Logger)
> space 21248K, 41% used [0x00000001102d0000, 0x0000000110b520b8,
> 0x0000000110b52200, 0x0000000111790000)
> No shared spaces configured.
>
> at java.util.logging.LogManager.addLogger(LogManager.java:399)
> - locked <0x000000011599a0b8> (a java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
> at com.sun.appserv.security.AppservRealm.<clinit>(AppservRealm.java:
> 64)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun
> .reflect
> .NativeConstructorAccessorImpl
> .newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingConstructorAccessorImpl
> .newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at java.lang.Class.newInstance0(Class.java:355)
> at java.lang.Class.newInstance(Class.java:308)
> at
> com
> .sun.enterprise.security.auth.realm.Realm.doInstantiate(Realm.java:
> 243)
> at
> com.sun.enterprise.security.auth.realm.Realm.instantiate(Realm.java:
> 165)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.loadRealms(RealmsMgrImpl.java:111)
> - locked <0x000000010b9c05c8> (a
> org.glassfish.admin.amx.impl.ext.RealmsMgrImpl)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.getRealm(RealmsMgrImpl.java:166)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.getUserNames(RealmsMgrImpl.java:252)
> at
> org
> .glassfish
> .admin
> .amx.impl.ext.RealmsMgrImpl.getAnonymousLogin(RealmsMgrImpl.java:360)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish
> .admin
> .amx.impl.mbean.AMXImplBase.getAttributeByMethod(AMXImplBase.java:588)
> at
> org
> .glassfish
> .admin
> .amx.impl.mbean.AMXImplBase.getAttributeInternal(AMXImplBase.java:505)
> at
> org
> .glassfish
> .admin.amx.impl.mbean.AMXImplBase.getAttribute(AMXImplBase.java:460)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .getAttribute(DefaultMBeanServerInterceptor.java:666)
> at
> com
> .sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
> 638)
> at
> org
> .glassfish
> .admin
> .amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:
> 251)
> at
> org
> .glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:
> 199)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 322)
> at
> org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
> 157)
> at
> org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
> 169)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
> 1082)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .invoke(DefaultMBeanServerInterceptor.java:836)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at
> javax
> .management
> .remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access
> $200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl
> $PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at
> javax
> .management
> .remote
> .rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:
> 1359)
> at
> javax
> .management
> .remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:
> 305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:
> 535)
> at sun.rmi.transport.tcp.TCPTransport
> $ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport
> $ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "RMI TCP Connection(idle)" daemon prio=5 tid=0x00000001032d2800
> nid=0x13b81f000 waiting on condition
> [0x000000013b81e000..0x000000013b81ead0]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001168cef70> (a
> java.util.concurrent.SynchronousQueue$TransferStack)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at java.util.concurrent.SynchronousQueue
> $TransferStack.awaitFulfill(SynchronousQueue.java:424)
> at java.util.concurrent.SynchronousQueue
> $TransferStack.transfer(SynchronousQueue.java:323)
> at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:
> 874)
> at
> java
> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:
> 944)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:906)
> at java.lang.Thread.run(Thread.java:637)
>
> "GrizzlySelectorRunner-TCP" prio=5 tid=0x00000001032c0800
> nid=0x13b413000 runnable [0x000000013b412000..0x000000013b412ad0]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x0000000116ac8878> (a sun.nio.ch.Util$1)
> - locked <0x0000000116ac8890> (a java.util.Collections
> $UnmodifiableSet)
> - locked <0x0000000116ac8800> (a sun.nio.ch.KQueueSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:489)
> at com.sun.grizzly.Controller.doSelect(Controller.java:328)
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:
> 90)
> at java.lang.Thread.run(Thread.java:637)
>
> "GrizzlySelectorRunner-TCP" prio=5 tid=0x00000001032c0000
> nid=0x13b310000 runnable [0x000000013b30f000..0x000000013b30fad0]
> java.lang.Thread.State: RUNNABLE
> at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
> at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
> at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked <0x0000000116ac8950> (a sun.nio.ch.Util$1)
> - locked <0x0000000116ac8968> (a java.util.Collections
> $UnmodifiableSet)
> - locked <0x0000000116ac88d8> (a sun.nio.ch.KQueueSelectorImpl)
> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> at
> com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:489)
> at com.sun.grizzly.Controller.doSelect(Controller.java:328)
> at
> com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:
> 90)
> at java.lang.Thread.run(Thread.java:637)
>
> "Thread-8" prio=5 tid=0x00000001032a8000 nid=0x13aa81000 in
> Object.wait() [0x000000013aa80000..0x000000013aa80ad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000116ab2228> (a
> java.util.concurrent.atomic.AtomicInteger)
> at
> com
> .sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:
> 1221)
> - locked <0x0000000116ab2228> (a
> java.util.concurrent.atomic.AtomicInteger)
> at com.sun.grizzly.Controller.start(Controller.java:968)
> at
> com
> .sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:
> 1180)
> - locked <0x0000000116ab2240> (a java.lang.Object)
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1031)
> at
> com
> .sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
> 1107)
> at com.sun.enterprise.v3.services.impl.GrizzlyProxy
> $1.run(GrizzlyProxy.java:211)
>
> "Thread-7" prio=5 tid=0x00000001032a4800 nid=0x13a46f000 in
> Object.wait() [0x000000013a46e000..0x000000013a46ead0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000116ab2500> (a
> java.util.concurrent.atomic.AtomicInteger)
> at
> com
> .sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:
> 1221)
> - locked <0x0000000116ab2500> (a
> java.util.concurrent.atomic.AtomicInteger)
> at com.sun.grizzly.Controller.start(Controller.java:968)
> at
> com
> .sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:
> 1180)
> - locked <0x0000000116ab2518> (a java.lang.Object)
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1031)
> at
> com
> .sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
> 1107)
> at com.sun.enterprise.v3.services.impl.GrizzlyProxy
> $1.run(GrizzlyProxy.java:211)
>
> "RMI RenewClean-[192.168.1.5:56351]" daemon prio=5
> tid=0x0000000103275800 nid=0x13b10a000 in Object.wait()
> [0x000000013b109000..0x000000013b109ad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011688b250> (a java.lang.ref.ReferenceQueue
> $Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x000000011688b250> (a java.lang.ref.ReferenceQueue$Lock)
> at sun.rmi.transport.DGCClient$EndpointEntry
> $RenewCleanThread.run(DGCClient.java:516)
> at java.lang.Thread.run(Thread.java:637)
>
> "RMI Scheduler(0)" daemon prio=5 tid=0x000000010326c000
> nid=0x13b007000 waiting on condition
> [0x000000013b006000..0x000000013b006ad0]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x000000011688b3b8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
> at
> java
> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:
> 946)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:906)
> at java.lang.Thread.run(Thread.java:637)
>
> "GC Daemon" daemon prio=2 tid=0x0000000103257800 nid=0x13a97e000 in
> Object.wait() [0x000000013a97d000..0x000000013a97dad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011688da78> (a sun.misc.GC$LatencyLock)
> at sun.misc.GC$Daemon.run(GC.java:100)
> - locked <0x000000011688da78> (a sun.misc.GC$LatencyLock)
>
> "RMI Reaper" prio=5 tid=0x0000000103253800 nid=0x13a87b000 in
> Object.wait() [0x000000013a87a000..0x000000013a87aad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011688db30> (a java.lang.ref.ReferenceQueue
> $Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x000000011688db30> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
> at java.lang.Thread.run(Thread.java:637)
>
> "RMI TCP Accept-0" daemon prio=5 tid=0x0000000103252000
> nid=0x13a778000 runnable [0x000000013a777000..0x000000013a777ad0]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> - locked <0x000000011688dc38> (a java.net.SocksSocketImpl)
> at java.net.ServerSocket.implAccept(ServerSocket.java:453)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.run(TCPTransport.java:341)
> at java.lang.Thread.run(Thread.java:637)
>
> "RMI TCP Accept-8686" daemon prio=5 tid=0x0000000103243000
> nid=0x13ae01000 runnable [0x000000013ae00000..0x000000013ae00ad0]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> - locked <0x000000011688dee8> (a java.net.SocksSocketImpl)
> at java.net.ServerSocket.implAccept(ServerSocket.java:453)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.run(TCPTransport.java:341)
> at java.lang.Thread.run(Thread.java:637)
>
> "transaction-manager" daemon prio=5 tid=0x000000010323e800
> nid=0x13acfe000 in Object.wait()
> [0x000000013acfd000..0x000000013acfdad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011688e190> (a java.util.TaskQueue)
> at java.lang.Object.wait(Object.java:485)
> at java.util.TimerThread.mainLoop(Timer.java:483)
> - locked <0x000000011688e190> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> "AutoDeployer" daemon prio=5 tid=0x000000010323e000 nid=0x13ab9b000
> in Object.wait() [0x000000013ab9a000..0x000000013ab9aad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011688e268> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:509)
> - locked <0x000000011688e268> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> "DynamicReloader" daemon prio=5 tid=0x0000000103201000
> nid=0x13a675000 in Object.wait()
> [0x000000013a674000..0x000000013a674ad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001167c0998> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:509)
> - locked <0x00000001167c0998> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> "pool-1-thread-5" prio=5 tid=0x00000001031fb800 nid=0x13a572000
> waiting on condition [0x000000013a571000..0x000000013a571ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001167c0aa8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "Thread-1" prio=5 tid=0x0000000103190800 nid=0x13a269000 waiting on
> condition [0x000000013a268000..0x000000013a268ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000116737908> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at
> com
> .sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:
> 498)
> at com.sun.enterprise.server.logging.GFFileHandler
> $3.run(GFFileHandler.java:246)
>
> "pool-2-thread-1" daemon prio=5 tid=0x000000010318f800
> nid=0x13a166000 waiting on condition
> [0x000000013a165000..0x000000013a165ad0]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001166d92d8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
> at
> java
> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:
> 946)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:906)
> at java.lang.Thread.run(Thread.java:637)
>
> "GlassFish Kernel Main Thread" prio=5 tid=0x0000000103185000
> nid=0x13a063000 in Object.wait()
> [0x000000013a062000..0x000000013a062ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001166d93c8> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
> at java.lang.Object.wait(Object.java:485)
> at com.sun.enterprise.v3.server.AppServerStartup
> $1.run(AppServerStartup.java:130)
> - locked <0x00000001166d93c8> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
>
> "pool-1-thread-3" prio=5 tid=0x0000000103161800 nid=0x139d94000
> waiting on condition [0x0000000139d93000..0x0000000139d93ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000116ebe978> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "pool-1-thread-2" prio=5 tid=0x000000010315e000 nid=0x139c3c000
> waiting on condition [0x0000000139c3b000..0x0000000139c3bad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001165dd240> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "pool-1-thread-1" prio=5 tid=0x000000010315d000 nid=0x139b39000
> waiting on condition [0x0000000139b38000..0x0000000139b38ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001165dd498> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "{felix.fileinstall.poll=5000,
> felix.fileinstall.bundles.new.start=false, felix.fileinstall.dir=/
> work/v3/glassfishv3/glassfish/modules/, felix.fileinstall.debug=1}"
> daemon prio=5 tid=0x00000001030fd000 nid=0x1392d6000 waiting on
> condition [0x00000001392d5000..0x00000001392d5ad0]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org
> .apache.felix.fileinstall.DirectoryWatcher.run(DirectoryWatcher.java:
> 122)
>
> "Configuration Updater" daemon prio=5 tid=0x00000001030fc000
> nid=0x1391d3000 in Object.wait()
> [0x00000001391d2000..0x00000001391d2ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000116185518> (a java.util.LinkedList)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
> - locked <0x0000000116185518> (a java.util.LinkedList)
>
> "telnetconsole.Listener" daemon prio=5 tid=0x00000001030f2000
> nid=0x1390d0000 runnable [0x00000001390cf000..0x00000001390cfad0]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> - locked <0x0000000116185600> (a java.net.SocksSocketImpl)
> at java.net.ServerSocket.implAccept(ServerSocket.java:453)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at org.apache.felix.shell.remote.Listener
> $Acceptor.run(Listener.java:137)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixPackageAdmin" daemon prio=5 tid=0x00000001030ea000
> nid=0x138fcd000 in Object.wait()
> [0x0000000138fcc000..0x0000000138fccad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001161594d8> (a
> org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Object.wait(Object.java:485)
> at
> org
> .apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:
> 314)
> - locked <0x00000001161594d8> (a
> org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixStartLevel" daemon prio=5 tid=0x00000001030e9800
> nid=0x138eca000 in Object.wait()
> [0x0000000138ec9000..0x0000000138ec9ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000116159cd0> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:241)
> - locked <0x0000000116159cd0> (a java.util.ArrayList)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixDispatchQueue" daemon prio=5 tid=0x000000010309e000
> nid=0x138969000 in Object.wait()
> [0x0000000138968000..0x0000000138968ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000115a10ce0> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at
> org
> .apache
> .felix.framework.util.EventDispatcher.run(EventDispatcher.java:875)
> - locked <0x0000000115a10ce0> (a java.util.ArrayList)
> at org.apache.felix.framework.util.EventDispatcher.access
> $000(EventDispatcher.java:50)
> at org.apache.felix.framework.util.EventDispatcher
> $1.run(EventDispatcher.java:102)
> at java.lang.Thread.run(Thread.java:637)
>
> "OSGi Framework Launcher" daemon prio=5 tid=0x0000000103082000
> nid=0x1380bd000 in Object.wait()
> [0x00000001380bb000..0x00000001380bcad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000011615b908> (a
> org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:
> 48)
> - locked <0x000000011615b908> (a
> org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.Felix.waitForStop(Felix.java:785)
> at org.apache.felix.main.Main.main(Main.java:215)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.enterprise.glassfish.bootstrap.ASMainFelix
> $1.run(ASMainFelix.java:91)
> at java.lang.Thread.run(Thread.java:637)
>
> "Low Memory Detector" daemon prio=5 tid=0x0000000103044800
> nid=0x137ad5000 runnable [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread1" daemon prio=9 tid=0x0000000103043000
> nid=0x1379d2000 waiting on condition
> [0x0000000000000000..0x00000001379d03e0]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=9 tid=0x0000000103041800
> nid=0x1378cf000 waiting on condition
> [0x0000000000000000..0x00000001378cd3e0]
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
> nid=0x1377cc000 waiting on condition
> [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x1376b2000 in
> Object.wait() [0x00000001376b1000..0x00000001376b1ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000115997518> (a java.lang.ref.ReferenceQueue
> $Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x0000000115997518> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> "Reference Handler" daemon prio=10 tid=0x0000000103028800
> nid=0x1375a9000 in Object.wait()
> [0x00000001375a8000..0x00000001375a8ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x0000000115997190> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x0000000115997190> (a java.lang.ref.Reference$Lock)
>
> "main" prio=5 tid=0x0000000103001000 nid=0x103901000 in
> Object.wait() [0x0000000103900000..0x0000000103900c40]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001159e6a00> (a java.lang.Thread)
> at java.lang.Thread.join(Thread.java:1161)
> - locked <0x00000001159e6a00> (a java.lang.Thread)
> at java.lang.Thread.join(Thread.java:1214)
> at
> com
> .sun
> .enterprise
> .glassfish.bootstrap.ASMainFelix.launchOSGiFW(ASMainFelix.java:110)
> at
> com
> .sun.enterprise.glassfish.bootstrap.ASMainOSGi.run(ASMainOSGi.java:
> 137)
> at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:65)
>
> "VM Thread" prio=9 tid=0x0000000103020800 nid=0x1374a6000 runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x0000000103046000
> nid=0x137bd8000 waiting on condition
>
> "Exception Catcher Thread" prio=10 tid=0x0000000103002000
> nid=0x103a04000 runnable
> JNI global references: 819
>
>
> Found one Java-level deadlock:
> =============================
> "ComplianceMonitor.ValidatorThread":
> waiting to lock monitor 0x0000000139935b98 (object
> 0x000000011599a0b8, a java.util.logging.LogManager),
> which is held by "RMI TCP Connection(4)-192.168.1.5"
> "RMI TCP Connection(4)-192.168.1.5":
> waiting to lock monitor 0x0000000139976ec8 (object
> 0x000000012b1c11b0, a java.lang.Class),
> which is held by "ComplianceMonitor.ValidatorThread"
>
> Java stack information for the threads listed above:
> ===================================================
> "ComplianceMonitor.ValidatorThread":
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000011599a0b8> (a
> java.util.logging.LogManager)
> at java.util.logging.Logger.getLogger(Logger.java:275)
> - locked <0x000000012b1c11b0> (a java.lang.Class for
> java.util.logging.Logger)
> at com.sun.jmx.trace.TraceManager.getLogger(TraceManager.java:166)
> at com.sun.jmx.trace.TraceManager.isSelected(TraceManager.java:197)
> at com.sun.jmx.trace.Trace.isSelected(Trace.java:84)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .isTraceOn(DefaultMBeanServerInterceptor.java:1832)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .getAttribute(DefaultMBeanServerInterceptor.java:657)
> at
> com
> .sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
> 638)
> at
> org
> .glassfish
> .admin
> .amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:
> 251)
> at
> org
> .glassfish
> .admin
> .amx
> .util
> .jmx.MBeanProxyHandler.getAttributeNoThrow(MBeanProxyHandler.java:263)
> at
> org
> .glassfish
> .admin
> .amx.core.proxy.AMXProxyHandler.getChildren(AMXProxyHandler.java:792)
> at
> org
> .glassfish
> .admin
> .amx.core.proxy.AMXProxyHandler.childrenSet(AMXProxyHandler.java:810)
> at
> org
> .glassfish
> .admin
> .amx
> .core.proxy.AMXProxyHandler.handleSpecialMethod(AMXProxyHandler.java:
> 421)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:
> 594)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
> at $Proxy66.childrenSet(Unknown Source)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 226)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 234)
> at
> org
> .glassfish
> .admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:
> 245)
> at
> org
> .glassfish
> .admin
> .amx.impl.path.PathnamesImpl.listObjectNames(PathnamesImpl.java:261)
> at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
> 1082)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .invoke(DefaultMBeanServerInterceptor.java:836)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at
> javax
> .management
> .MBeanServerInvocationHandler
> .invoke(MBeanServerInvocationHandler.java:288)
> at
> org
> .glassfish
> .admin.amx.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:
> 467)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:
> 605)
> at
> org
> .glassfish
> .admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
> at $Proxy69.listObjectNames(Unknown Source)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 318)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 331)
> at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
> $ValidatorThread.doRun(ComplianceMonitor.java:134)
> at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
> $ValidatorThread.run(ComplianceMonitor.java:118)
> "RMI TCP Connection(4)-192.168.1.5":
> at java.util.logging.Logger.getLogger(Logger.java:274)
> - waiting to lock <0x000000012b1c11b0> (a java.lang.Class for
> java.util.logging.Logger)
> at java.util.logging.LogManager.addLogger(LogManager.java:399)
> - locked <0x000000011599a0b8> (a java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
> at com.sun.appserv.security.AppservRealm.<clinit>(AppservRealm.java:
> 64)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun
> .reflect
> .NativeConstructorAccessorImpl
> .newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingConstructorAccessorImpl
> .newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at java.lang.Class.newInstance0(Class.java:355)
> at java.lang.Class.newInstance(Class.java:308)
> at
> com
> .sun.enterprise.security.auth.realm.Realm.doInstantiate(Realm.java:
> 243)
> at
> com.sun.enterprise.security.auth.realm.Realm.instantiate(Realm.java:
> 165)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.loadRealms(RealmsMgrImpl.java:111)
> - locked <0x000000010b9c05c8> (a
> org.glassfish.admin.amx.impl.ext.RealmsMgrImpl)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.getRealm(RealmsMgrImpl.java:166)
> at
> org
> .glassfish
> .admin.amx.impl.ext.RealmsMgrImpl.getUserNames(RealmsMgrImpl.java:252)
> at
> org
> .glassfish
> .admin
> .amx.impl.ext.RealmsMgrImpl.getAnonymousLogin(RealmsMgrImpl.java:360)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish
> .admin
> .amx.impl.mbean.AMXImplBase.getAttributeByMethod(AMXImplBase.java:588)
> at
> org
> .glassfish
> .admin
> .amx.impl.mbean.AMXImplBase.getAttributeInternal(AMXImplBase.java:505)
> at
> org
> .glassfish
> .admin.amx.impl.mbean.AMXImplBase.getAttribute(AMXImplBase.java:460)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .getAttribute(DefaultMBeanServerInterceptor.java:666)
> at
> com
> .sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
> 638)
> at
> org
> .glassfish
> .admin
> .amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:
> 251)
> at
> org
> .glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:
> 199)
> at
> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
> 322)
> at
> org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
> 157)
> at
> org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
> 169)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org
> .glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
> 1082)
> at
> com
> .sun
> .jmx
> .interceptor
> .DefaultMBeanServerInterceptor
> .invoke(DefaultMBeanServerInterceptor.java:836)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
> at
> javax
> .management
> .remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
> at javax.management.remote.rmi.RMIConnectionImpl.access
> $200(RMIConnectionImpl.java:72)
> at javax.management.remote.rmi.RMIConnectionImpl
> $PrivilegedOperation.run(RMIConnectionImpl.java:1264)
> at
> javax
> .management
> .remote
> .rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:
> 1359)
> at
> javax
> .management
> .remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:
> 305)
> at sun.rmi.transport.Transport$1.run(Transport.java:159)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
> at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:
> 535)
> at sun.rmi.transport.tcp.TCPTransport
> $ConnectionHandler.run0(TCPTransport.java:790)
> at sun.rmi.transport.tcp.TCPTransport
> $ConnectionHandler.run(TCPTransport.java:649)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> Found 1 deadlock.
>
> Heap
> def new generation total 13056K, used 3194K [0x000000010aed0000,
> 0x000000010bd60000, 0x0000000115970000)
> eden space 11200K, 20% used [0x000000010aed0000,
> 0x000000010b1141d0, 0x000000010b9c0000)
> from space 1856K, 47% used [0x000000010b9c0000,
> 0x000000010ba9a7f8, 0x000000010bb90000)
> to space 1856K, 0% used [0x000000010bb90000,
> 0x000000010bb90000, 0x000000010bd60000)
> tenured generation total 29580K, used 23266K [0x0000000115970000,
> 0x0000000117653000, 0x000000012aed0000)
> the space 29580K, 78% used [0x0000000115970000,
> 0x0000000117028970, 0x0000000117028a00, 0x0000000117653000)
> compacting perm gen total 25408K, used 25108K [0x000000012aed0000,
> 0x000000012c7a0000, 0x0000000136ed0000)
> the space 25408K, 98% used [0x000000012aed0000,
> 0x000000012c7552d0, 0x000000012c755400, 0x000000012c7a0000)
> No shared spaces configured.
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>

Lloyd Chambers
lloyd.chambers_at_sun.com
GlassFish Team