dev@glassfish.java.net

deadlock in LogManager

From: Lloyd Chambers <Lloyd.Chambers_at_Sun.COM>
Date: Fri, 24 Apr 2009 12:20:47 -0700

While testing some code today, I came across this deadlock in
LogManager.

Is this a JDK bug, or a bug due to something we've inserted into the
logging mechanism?



^CThe domain was stopped.
llcMP:code llc$ r3

Apr 24, 2009 12:14:51 PM
com.sun.enterprise.admin.launcher.GFLauncherLogger info
INFO: JVM invocation command line:
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/bin/java
-cp
/work/v3/glassfishv3/glassfish/modules/glassfish.jar
-XX:+UnlockDiagnosticVMOptions
-XX:MaxPermSize=192m
-XX:NewRatio=2
-XX:+LogVMOutput
-XX:LogFile=/work/v3/glassfishv3/glassfish/domains/domain1/logs/jvm.log
-Xmx512m
-client
-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver
-Djavax.net.ssl.trustStore=/work/v3/glassfishv3/glassfish/domains/
domain1/config/cacerts.jks
-Djavax.net.ssl.keyStore=/work/v3/glassfishv3/glassfish/domains/
domain1/config/keystore.jks
-Djava.security.policy=/work/v3/glassfishv3/glassfish/domains/domain1/
config/server.policy
-Dcom.sun.aas.instanceRoot=/work/v3/glassfishv3/glassfish/domains/
domain1
-
Dcom
.sun
.enterprise
.config
.config_environment_factory_class
=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory
-Djava.security.auth.login.config=/work/v3/glassfishv3/glassfish/
domains/domain1/config/login.conf
-Djava.endorsed.dirs=/work/v3/glassfishv3/glassfish/lib/endorsed
-Dcom.sun.aas.installRoot=/work/v3/glassfishv3/glassfish
-Djava.ext.dirs=/System/Library/Frameworks/JavaVM.framework/Versions/
1.6.0/Home/lib/ext:/System/Library/Frameworks/JavaVM.framework/
Versions/1.6.0/Home/jre/lib/ext:/work/v3/glassfishv3/glassfish/domains/
domain1/lib/ext:/work/v3/glassfishv3/javadb/lib
com.sun.enterprise.glassfish.bootstrap.ASMain
-domainname
domain1
-asadmin-args
start-domain,,,--verbose
-instancename
server
-watchdog
true
-verbose
true
-debug
false
-asadmin-classpath
/v3/glassfishv3/bin/../glassfish/modules/admin-cli.jar
-asadmin-classname
com.sun.enterprise.admin.cli.AsadminMain
-domaindir
/work/v3/glassfishv3/glassfish/domains/domain1
Watchdog is running. The Domain is remote-restart enabled.
Apr 24, 2009 12:14:51 PM
com.sun.enterprise.admin.launcher.GFLauncherLogger info
INFO: Successfully launched in 13 msec.
Apr 24, 2009 12:14:51 PM com.sun.enterprise.glassfish.bootstrap.ASMain
main
INFO: Launching GlassFish on Apache Felix OSGi platform

Welcome to Felix.
=================

felix.fileinstall.poll (ms) 5000
felix.fileinstall.dir /work/v3/glassfishv3/glassfish/modules
felix.fileinstall.debug 1
felix.fileinstall.bundles.new.start false
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-mbeanserver [103]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.core.kernel [164]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.util [48]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle com.sun.enterprise.config [53]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.admin.config-api [52]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.branding [37]
Apr 24, 2009 12:14:52 PM AppServerStartup run
INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
Apr 24, 2009 12:14:52 PM
com.sun.enterprise.v3.services.impl.LogManagerService postConstruct
INFO: Starting GlassFish Application Server.
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.internal-api [127]
Apr 24, 2009 12:14:52 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.registration.glassfish-registration
[106]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.jms.core [148]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.connectors.runtime [57]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-naming [105]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jta [162]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.glassfish-api [91]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.container-common [67]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.ejb.ejb-container [77]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.autodeploy [71]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.transaction.jts [163]
Apr 24, 2009 12:14:53 PM
com.sun.enterprise.transaction.JavaEETransactionManagerSimplified
initDelegates
INFO: Using
com.sun.enterprise.transaction.jts.JavaEETransactionManagerJTSDelegate
as the delegate
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.connectors.work-management [216]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.connectors.internal-api [56]
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.flashlight.framework [82]
Apr 24, 2009 12:14:53 PM
org.glassfish.admin.mbeanserver.ConnectorStartupService
$ConnectorsStarterThread startConnector
INFO: Started JMXConnector, JMXService URL = service:jmx:rmi:///jndi/
rmi://192.168.1.5:8686/jmxrmi
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle com.sun.grizzly.grizzly-config [111]
Apr 24, 2009 12:14:53 PM
com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 8080
Apr 24, 2009 12:14:53 PM
com.sun.enterprise.v3.services.impl.GrizzlyService createNetworkProxy
INFO: Network listener http-listener-2 on port 8181 disabled per
domain.xml
Apr 24, 2009 12:14:53 PM
com.sun.enterprise.v3.services.impl.GrizzlyProxy start
INFO: Listening on port 4848
Apr 24, 2009 12:14:53 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.deployment.common [73]
Apr 24, 2009 12:14:53 PM com.sun.enterprise.v3.server.AppServerStartup
run
INFO: GlassFish v3 startup time : Felix(1253ms) startup
services(693ms) total(1946ms)
Apr 24, 2009 12:14:56 PM
org.glassfish.admin.mbeanserver.ConnectorStartupService
$BootAMXListener handleNotification
INFO: ConnectorStartupService.BootAMXListener: connection made,
booting AMX MBeans
Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.amx-impl [19]
Apr 24, 2009 12:14:56 PM
INFO: org.glassfish.admin.amx.loader.AMXStartupService
Apr 24, 2009 12:14:56 PM
INFO: Got loader for AMXStartupServiceMBean:
org.glassfish.admin.amx.loader.AMXStartupService_at_237dabb0
Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.security.realms [187]
Apr 24, 2009 12:14:56 PM
org.glassfish.admin.amx.loader.AMXStartupService startAMX
INFO: AMXStartupService: loaded 117 AMX MBeans in 315 ms (wall-clock
time)
Apr 24, 2009 12:14:56 PM
org.glassfish.admin.mbeanserver.ConnectorStartupService
$BootAMXListener handleNotification
INFO: ConnectorStartupService.BootAMXListener: connection made,
booting AMX MBeans
Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.amx-core-impl [13]
Apr 24, 2009 12:14:56 PM
INFO: ProxyFactory: mDomainRootObjectName =
v3:pp=,type=DomainRoot,name=v3
Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.amx-j2ee-impl [21]
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/,type=SystemInfo
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/,type=J2EEDomain,j2eeType=J2EEDomain,name=v3
Apr 24, 2009 12:14:56 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.amx-config-impl [9]
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/
J2EEDomain,type=J2EEServer,name=das,j2eeType=J2EEServer
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/J2EEDomain/
J2EEServer[das],type=JVM,j2eeType=JVM,J2EEServer=das
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/,type=domain
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain,type=system-applications
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain,type=applications
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain,type=resources
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-resource,name=jdbc/
__TimerPool
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-resource,name=jdbc/
__default
Apr 24, 2009 12:14:56 PM
INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-connection-
pool,name=__TimerPool
Apr 24, 2009 12:14:57 PM
org.glassfish.admin.amx.impl.config.ConfigBeanJMXSupport <init>
WARNING: ConfigBeanJMXSupport (AMX): working around @Configured bug
for com.sun.grizzly.config.dom.ThreadPool, using "ThreadPoolId" as the
key attribute
Apr 24, 2009 12:14:57 PM OSGiModuleImpl start
INFO: Started bundle org.glassfish.common.amx-ext-impl [17]
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
pool[__TimerPool],type=property,name=databaseName
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
pool[__TimerPool],type=property,name=connectionAttributes
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources,type=jdbc-connection-
pool,name=DerbyPool
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
pool[DerbyPool],type=property,name=PortNumber
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
pool[DerbyPool],type=property,name=Password
Apr 24, 2009 12:14:57 PM
INFO: VALIDATED: v3:pp=/domain/resources/jdbc-connection-
pool[DerbyPool],type=property,name=User



^\2009-04-24 12:18:30
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
mixed mode):

"domain1-StderrDrainer" daemon prio=5 tid=0x00000001030cd800
nid=0x116d76000 runnable [0x0000000116d75000..0x0000000116d75ad0]
    java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:199)
        at java.lang.UNIXProcess
$DeferredCloseInputStream.read(UNIXProcess.java:227)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        - locked <0x000000010b113820> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read(FilterInputStream.java:90)
        at
com
.sun
.enterprise
.universal
.process
.ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:69)
        at java.lang.Thread.run(Thread.java:637)

"domain1-StdoutDrainer" daemon prio=5 tid=0x00000001030cd000
nid=0x116c73000 runnable [0x0000000116c72000..0x0000000116c72ad0]
    java.lang.Thread.State: RUNNABLE
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:199)
        at java.lang.UNIXProcess
$DeferredCloseInputStream.read(UNIXProcess.java:227)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        - locked <0x000000010b113a48> (a java.io.BufferedInputStream)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:258)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
        - locked <0x000000010b1138f8> (a java.io.BufferedInputStream)
        at java.io.FilterInputStream.read(FilterInputStream.java:90)
        at
com
.sun
.enterprise
.universal
.process
.ProcessStreamDrainerWorker.run(ProcessStreamDrainerWorker.java:69)
        at java.lang.Thread.run(Thread.java:637)

"process reaper" daemon prio=5 tid=0x00000001030cc000 nid=0x116b70000
runnable [0x0000000116b6f000..0x0000000116b6fad0]
    java.lang.Thread.State: RUNNABLE
        at java.lang.UNIXProcess.waitForProcessExit(Native Method)
        at java.lang.UNIXProcess.access$900(UNIXProcess.java:17)
        at java.lang.UNIXProcess$2$1.run(UNIXProcess.java:86)

"Low Memory Detector" daemon prio=5 tid=0x0000000103044800
nid=0x116105000 runnable [0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=0x0000000103043000 nid=0x116002000
waiting on condition [0x0000000000000000..0x00000001160003e0]
    java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=0x0000000103041800 nid=0x115eff000
waiting on condition [0x0000000000000000..0x0000000115efd3e0]
    java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
nid=0x115dfc000 waiting on condition
[0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x115ce2000 in
Object.wait() [0x0000000115ce1000..0x0000000115ce1ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010cafb828> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x000000010cafb828> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000103028800
nid=0x115bd9000 in Object.wait()
[0x0000000115bd8000..0x0000000115bd8ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010cafb908> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x000000010cafb908> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x0000000103001000 nid=0x103901000 in Object.wait()
[0x0000000103900000..0x0000000103900c70]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000010b110250> (a java.lang.UNIXProcess)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.UNIXProcess.waitFor(UNIXProcess.java:115)
        - locked <0x000000010b110250> (a java.lang.UNIXProcess)
        at com.sun.enterprise.admin.launcher.GFLauncher.wait(GFLauncher.java:
298)
        at
com
.sun
.enterprise.admin.launcher.GFLauncher.launchInstance(GFLauncher.java:
248)
        at
com
.sun
.enterprise
.admin.launcher.GFDomainLauncher.internalLaunch(GFDomainLauncher.java:
47)
        at
com.sun.enterprise.admin.launcher.GFLauncher.launch(GFLauncher.java:68)
        - locked <0x000000010cb499e0> (a
com.sun.enterprise.admin.launcher.GFDomainLauncher)
        at
com
.sun
.enterprise
.admin
.cli.StartDomainCommand.runCommandNotEmbedded(StartDomainCommand.java:
99)
        at
com
.sun
.enterprise
.admin.cli.StartDomainCommand.runCommand(StartDomainCommand.java:46)
        at
com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:160)
        at com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:150)
        at com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:76)

"VM Thread" prio=9 tid=0x0000000103020800 nid=0x115ad6000 runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000103046000
nid=0x116208000 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x0000000103002000
nid=0x103a04000 runnable
2009-04-24 12:18:30
Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
mixed mode):

"AMXConfigLoader.AMXConfigLoaderThread" daemon prio=5
tid=0x0000000103321000 nid=0x13bc2b000 waiting on condition
[0x000000013bc2a000..0x000000013bc2aad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116623cf0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
JNI global references: 657

Heap
  def new generation total 2304K, used 298K [0x000000010aed0000,
0x000000010b160000, 0x000000010cad0000)
   eden space 1984K, 2% used at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:
358)
        at
org
.glassfish
.admin
.mbeanserver.PendingConfigBeansNew.take(PendingConfigBeansNew.java:115)
        at org.glassfish.admin.amx.impl.config.AMXConfigLoader
$AMXConfigLoaderThread.doRun(AMXConfigLoader.java:482)
        at org.glassfish.admin.amx.impl.config.AMXConfigLoader
$AMXConfigLoaderThread.run(AMXConfigLoader.java:448)

"ComplianceMonitor.ValidatorThread" daemon prio=5
tid=0x0000000103317800 nid=0x13bb28000 waiting for monitor entry
[0x000000013bb26000..0x000000013bb27ad0]
    java.lang.Thread.State: BLOCKED (on object monitor)
        at java.util.logging.LogManager.getLogger(LogManager.java:550)
        - waiting to lock <0x000000011599a0b8> (a java.util.logging.LogManager)
        at java.util.logging.Logger.getLogger(Logger.java:275)
        - locked <0x000000012b1c11b0> (a java.lang.Class for
java.util.logging.Logger)
        at com.sun.jmx.trace.TraceManager.getLogger(TraceManager.java:166)
        at com.sun.jmx.trace.TraceManager.isSelected(TraceManager.java:197)
        at com.sun.jmx.trace.Trace.isSelected(Trace.java:84)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.isTraceOn(DefaultMBeanServerInterceptor.java:1832)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.getAttribute(DefaultMBeanServerInterceptor.java:657)
        at
com
.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
638)
        at
org
.glassfish
.admin
.amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:251)
        at
org
.glassfish
.admin
.amx
.util.jmx.MBeanProxyHandler.getAttributeNoThrow(MBeanProxyHandler.java:
263)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.getChildren(AMXProxyHandler.java:
792)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.childrenSet(AMXProxyHandler.java:
810)
        at
org
.glassfish
.admin
.amx
.core.proxy.AMXProxyHandler.handleSpecialMethod(AMXProxyHandler.java:
421)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:594)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
        at $Proxy66.childrenSet(Unknown Source)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:226)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:234)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:245)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listObjectNames(PathnamesImpl.java:
261)
        at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
1082)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:
761)
        at
javax
.management
.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:
288)
        at
org
.glassfish
.admin.amx.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:467)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:605)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
        at $Proxy69.listObjectNames(Unknown Source)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
318)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
331)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
$ValidatorThread.doRun(ComplianceMonitor.java:134)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
$ValidatorThread.run(ComplianceMonitor.java:118)

"pool-1-thread-6" prio=5 tid=0x00000001032f3800 nid=0x13c168000
waiting on condition [0x000000013c167000..0x000000013c167ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116ac5278> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.con [0x000000010aed0000, 0x000000010aedb240,
0x000000010b0c0000)
   from space 320K, 79% used [0x000000010b110000, 0x000000010b14f690,
0x000000010b160000)
   to space 320K, 0% used [0x000000010b0c0000, 0x000000010b0c0000,
0x000000010b110000)
  tenured generation total 5312K, used 735K [0x000000010cad0000,
0x000000010d000000, 0x00000001102d0000)
    the space 5312K, 13% used [0x000000010cad0000,
0x000000010cb87fc0, 0x000000010cb88000, 0x000000010d000000)
  compacting perm gen total 21248K, used 8712K [0x00000001102d0000,
0x0000000111790000, 0x00000001156d0000)
    thecurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"AMXConfigLoader.AMXConfigLoaderThread" daemon prio=5
tid=0x00000001032f3000 nid=0x13c065000 waiting on condition
[0x000000013c064000..0x000000013c064ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116641c68> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:
358)
        at
org
.glassfish
.admin.mbeanserver.PendingConfigBeans.take(PendingConfigBeans.java:117)
        at org.glassfish.admin.amx.config.AMXConfigLoader
$AMXConfigLoaderThread.doRun(AMXConfigLoader.java:521)
        at org.glassfish.admin.amx.config.AMXConfigLoader
$AMXConfigLoaderThread.run(AMXConfigLoader.java:488)

"JMX server connection timeout 51" daemon prio=5
tid=0x00000001032d4000 nid=0x13ba25000 in Object.wait()
[0x000000013ba24000..0x000000013ba24ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000116ac55f0> (a [I)
        at java.lang.Object.wait(Object.java:485)
        at com.sun.jmx.remote.internal.ServerCommunicatorAdmin
$Timeout.run(ServerCommunicatorAdmin.java:132)
        - locked <0x0000000116ac55f0> (a [I)
        at java.lang.Thread.run(Thread.java:637)

"RMI TCP Connection(4)-192.168.1.5" daemon prio=5
tid=0x00000001032d3800 nid=0x13b922000 waiting for monitor entry
[0x000000013b91d000..0x000000013b921ad0]
    java.lang.Thread.State: BLOCKED (on object monitor)
        at java.util.logging.Logger.getLogger(Logger.java:274)
        - waiting to lock <0x000000012b1c11b0> (a java.lang.Class for
java.util.logging.Logger)
  space 21248K, 41% used [0x00000001102d0000, 0x0000000110b520b8,
0x0000000110b52200, 0x0000000111790000)
No shared spaces configured.

        at java.util.logging.LogManager.addLogger(LogManager.java:399)
        - locked <0x000000011599a0b8> (a java.util.logging.LogManager)
        at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
        at com.sun.appserv.security.AppservRealm.<clinit>(AppservRealm.java:64)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at
com.sun.enterprise.security.auth.realm.Realm.doInstantiate(Realm.java:
243)
        at
com.sun.enterprise.security.auth.realm.Realm.instantiate(Realm.java:165)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.loadRealms(RealmsMgrImpl.java:111)
        - locked <0x000000010b9c05c8> (a
org.glassfish.admin.amx.impl.ext.RealmsMgrImpl)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getRealm(RealmsMgrImpl.java:166)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getUserNames(RealmsMgrImpl.java:252)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getAnonymousLogin(RealmsMgrImpl.java:
360)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org
.glassfish
.admin
.amx.impl.mbean.AMXImplBase.getAttributeByMethod(AMXImplBase.java:588)
        at
org
.glassfish
.admin
.amx.impl.mbean.AMXImplBase.getAttributeInternal(AMXImplBase.java:505)
        at
org
.glassfish
.admin.amx.impl.mbean.AMXImplBase.getAttribute(AMXImplBase.java:460)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.getAttribute(DefaultMBeanServerInterceptor.java:666)
        at
com
.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
638)
        at
org
.glassfish
.admin
.amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:251)
        at
org.glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:
199)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
322)
        at
org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
157)
        at
org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
169)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
1082)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:
761)
        at
javax
.management
.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
        at javax.management.remote.rmi.RMIConnectionImpl.access
$200(RMIConnectionImpl.java:72)
        at javax.management.remote.rmi.RMIConnectionImpl
$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
        at
javax
.management
.remote
.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:
1359)
        at
javax
.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:
788)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
        at sun.rmi.transport.Transport$1.run(Transport.java:159)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
        at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
        at sun.rmi.transport.tcp.TCPTransport
$ConnectionHandler.run0(TCPTransport.java:790)
        at sun.rmi.transport.tcp.TCPTransport
$ConnectionHandler.run(TCPTransport.java:649)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"RMI TCP Connection(idle)" daemon prio=5 tid=0x00000001032d2800
nid=0x13b81f000 waiting on condition
[0x000000013b81e000..0x000000013b81ead0]
    java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000001168cef70> (a
java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:
198)
        at java.util.concurrent.SynchronousQueue
$TransferStack.awaitFulfill(SynchronousQueue.java:424)
        at java.util.concurrent.SynchronousQueue
$TransferStack.transfer(SynchronousQueue.java:323)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:
874)
        at
java
.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:944)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:906)
        at java.lang.Thread.run(Thread.java:637)

"GrizzlySelectorRunner-TCP" prio=5 tid=0x00000001032c0800
nid=0x13b413000 runnable [0x000000013b412000..0x000000013b412ad0]
    java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0x0000000116ac8878> (a sun.nio.ch.Util$1)
        - locked <0x0000000116ac8890> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0000000116ac8800> (a sun.nio.ch.KQueueSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:
489)
        at com.sun.grizzly.Controller.doSelect(Controller.java:328)
        at
com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:90)
        at java.lang.Thread.run(Thread.java:637)

"GrizzlySelectorRunner-TCP" prio=5 tid=0x00000001032c0000
nid=0x13b310000 runnable [0x000000013b30f000..0x000000013b30fad0]
    java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
        at sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:136)
        at sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:69)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0x0000000116ac8950> (a sun.nio.ch.Util$1)
        - locked <0x0000000116ac8968> (a java.util.Collections$UnmodifiableSet)
        - locked <0x0000000116ac88d8> (a sun.nio.ch.KQueueSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:
489)
        at com.sun.grizzly.Controller.doSelect(Controller.java:328)
        at
com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:90)
        at java.lang.Thread.run(Thread.java:637)

"Thread-8" prio=5 tid=0x00000001032a8000 nid=0x13aa81000 in
Object.wait() [0x000000013aa80000..0x000000013aa80ad0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000116ab2228> (a
java.util.concurrent.atomic.AtomicInteger)
        at
com
.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:
1221)
        - locked <0x0000000116ab2228> (a
java.util.concurrent.atomic.AtomicInteger)
        at com.sun.grizzly.Controller.start(Controller.java:968)
        at
com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:
1180)
        - locked <0x0000000116ab2240> (a java.lang.Object)
        at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1031)
        at
com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
1107)
        at com.sun.enterprise.v3.services.impl.GrizzlyProxy
$1.run(GrizzlyProxy.java:211)

"Thread-7" prio=5 tid=0x00000001032a4800 nid=0x13a46f000 in
Object.wait() [0x000000013a46e000..0x000000013a46ead0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000116ab2500> (a
java.util.concurrent.atomic.AtomicInteger)
        at
com
.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:
1221)
        - locked <0x0000000116ab2500> (a
java.util.concurrent.atomic.AtomicInteger)
        at com.sun.grizzly.Controller.start(Controller.java:968)
        at
com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:
1180)
        - locked <0x0000000116ab2518> (a java.lang.Object)
        at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1031)
        at
com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
1107)
        at com.sun.enterprise.v3.services.impl.GrizzlyProxy
$1.run(GrizzlyProxy.java:211)

"RMI RenewClean-[192.168.1.5:56351]" daemon prio=5
tid=0x0000000103275800 nid=0x13b10a000 in Object.wait()
[0x000000013b109000..0x000000013b109ad0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011688b250> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x000000011688b250> (a java.lang.ref.ReferenceQueue$Lock)
        at sun.rmi.transport.DGCClient$EndpointEntry
$RenewCleanThread.run(DGCClient.java:516)
        at java.lang.Thread.run(Thread.java:637)

"RMI Scheduler(0)" daemon prio=5 tid=0x000000010326c000
nid=0x13b007000 waiting on condition
[0x000000013b006000..0x000000013b006ad0]
    java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x000000011688b3b8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:
198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
        at
java
.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:906)
        at java.lang.Thread.run(Thread.java:637)

"GC Daemon" daemon prio=2 tid=0x0000000103257800 nid=0x13a97e000 in
Object.wait() [0x000000013a97d000..0x000000013a97dad0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011688da78> (a sun.misc.GC$LatencyLock)
        at sun.misc.GC$Daemon.run(GC.java:100)
        - locked <0x000000011688da78> (a sun.misc.GC$LatencyLock)

"RMI Reaper" prio=5 tid=0x0000000103253800 nid=0x13a87b000 in
Object.wait() [0x000000013a87a000..0x000000013a87aad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011688db30> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x000000011688db30> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at sun.rmi.transport.ObjectTable$Reaper.run(ObjectTable.java:333)
        at java.lang.Thread.run(Thread.java:637)

"RMI TCP Accept-0" daemon prio=5 tid=0x0000000103252000
nid=0x13a778000 runnable [0x000000013a777000..0x000000013a777ad0]
    java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        - locked <0x000000011688dc38> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at sun.rmi.transport.tcp.TCPTransport
$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at sun.rmi.transport.tcp.TCPTransport
$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:637)

"RMI TCP Accept-8686" daemon prio=5 tid=0x0000000103243000
nid=0x13ae01000 runnable [0x000000013ae00000..0x000000013ae00ad0]
    java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        - locked <0x000000011688dee8> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at sun.rmi.transport.tcp.TCPTransport
$AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
        at sun.rmi.transport.tcp.TCPTransport
$AcceptLoop.run(TCPTransport.java:341)
        at java.lang.Thread.run(Thread.java:637)

"transaction-manager" daemon prio=5 tid=0x000000010323e800
nid=0x13acfe000 in Object.wait()
[0x000000013acfd000..0x000000013acfdad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011688e190> (a java.util.TaskQueue)
        at java.lang.Object.wait(Object.java:485)
        at java.util.TimerThread.mainLoop(Timer.java:483)
        - locked <0x000000011688e190> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"AutoDeployer" daemon prio=5 tid=0x000000010323e000 nid=0x13ab9b000 in
Object.wait() [0x000000013ab9a000..0x000000013ab9aad0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011688e268> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0x000000011688e268> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"DynamicReloader" daemon prio=5 tid=0x0000000103201000 nid=0x13a675000
in Object.wait() [0x000000013a674000..0x000000013a674ad0]
    java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001167c0998> (a java.util.TaskQueue)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0x00000001167c0998> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

"pool-1-thread-5" prio=5 tid=0x00000001031fb800 nid=0x13a572000
waiting on condition [0x000000013a571000..0x000000013a571ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000001167c0aa8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"Thread-1" prio=5 tid=0x0000000103190800 nid=0x13a269000 waiting on
condition [0x000000013a268000..0x000000013a268ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116737908> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at
com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:
498)
        at com.sun.enterprise.server.logging.GFFileHandler
$3.run(GFFileHandler.java:246)

"pool-2-thread-1" daemon prio=5 tid=0x000000010318f800 nid=0x13a166000
waiting on condition [0x000000013a165000..0x000000013a165ad0]
    java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000001166d92d8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:
198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
        at java.util.concurrent.ScheduledThreadPoolExecutor
$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
        at
java
.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:906)
        at java.lang.Thread.run(Thread.java:637)

"GlassFish Kernel Main Thread" prio=5 tid=0x0000000103185000
nid=0x13a063000 in Object.wait()
[0x000000013a062000..0x000000013a062ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001166d93c8> (a
com.sun.enterprise.v3.server.AppServerStartup$1)
        at java.lang.Object.wait(Object.java:485)
        at com.sun.enterprise.v3.server.AppServerStartup
$1.run(AppServerStartup.java:130)
        - locked <0x00000001166d93c8> (a
com.sun.enterprise.v3.server.AppServerStartup$1)

"pool-1-thread-3" prio=5 tid=0x0000000103161800 nid=0x139d94000
waiting on condition [0x0000000139d93000..0x0000000139d93ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x0000000116ebe978> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"pool-1-thread-2" prio=5 tid=0x000000010315e000 nid=0x139c3c000
waiting on condition [0x0000000139c3b000..0x0000000139c3bad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000001165dd240> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"pool-1-thread-1" prio=5 tid=0x000000010315d000 nid=0x139b39000
waiting on condition [0x0000000139b38000..0x0000000139b38ad0]
    java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x00000001165dd498> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer
$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at
java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:
142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:
441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

"{felix.fileinstall.poll=5000,
felix.fileinstall.bundles.new.start=false, felix.fileinstall.dir=/work/
v3/glassfishv3/glassfish/modules/, felix.fileinstall.debug=1}" daemon
prio=5 tid=0x00000001030fd000 nid=0x1392d6000 waiting on condition
[0x00000001392d5000..0x00000001392d5ad0]
    java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at
org
.apache.felix.fileinstall.DirectoryWatcher.run(DirectoryWatcher.java:
122)

"Configuration Updater" daemon prio=5 tid=0x00000001030fc000
nid=0x1391d3000 in Object.wait()
[0x00000001391d2000..0x00000001391d2ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000116185518> (a java.util.LinkedList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
        - locked <0x0000000116185518> (a java.util.LinkedList)

"telnetconsole.Listener" daemon prio=5 tid=0x00000001030f2000
nid=0x1390d0000 runnable [0x00000001390cf000..0x00000001390cfad0]
    java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
        - locked <0x0000000116185600> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:453)
        at java.net.ServerSocket.accept(ServerSocket.java:421)
        at org.apache.felix.shell.remote.Listener$Acceptor.run(Listener.java:
137)
        at java.lang.Thread.run(Thread.java:637)

"FelixPackageAdmin" daemon prio=5 tid=0x00000001030ea000
nid=0x138fcd000 in Object.wait()
[0x0000000138fcc000..0x0000000138fccad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001161594d8> (a
org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Object.wait(Object.java:485)
        at
org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:
314)
        - locked <0x00000001161594d8> (a
org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Thread.run(Thread.java:637)

"FelixStartLevel" daemon prio=5 tid=0x00000001030e9800 nid=0x138eca000
in Object.wait() [0x0000000138ec9000..0x0000000138ec9ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000116159cd0> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:
241)
        - locked <0x0000000116159cd0> (a java.util.ArrayList)
        at java.lang.Thread.run(Thread.java:637)

"FelixDispatchQueue" daemon prio=5 tid=0x000000010309e000
nid=0x138969000 in Object.wait()
[0x0000000138968000..0x0000000138968ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115a10ce0> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at
org
.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:
875)
        - locked <0x0000000115a10ce0> (a java.util.ArrayList)
        at org.apache.felix.framework.util.EventDispatcher.access
$000(EventDispatcher.java:50)
        at org.apache.felix.framework.util.EventDispatcher
$1.run(EventDispatcher.java:102)
        at java.lang.Thread.run(Thread.java:637)

"OSGi Framework Launcher" daemon prio=5 tid=0x0000000103082000
nid=0x1380bd000 in Object.wait()
[0x00000001380bb000..0x00000001380bcad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x000000011615b908> (a
org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:48)
        - locked <0x000000011615b908> (a
org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.Felix.waitForStop(Felix.java:785)
        at org.apache.felix.main.Main.main(Main.java:215)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.sun.enterprise.glassfish.bootstrap.ASMainFelix
$1.run(ASMainFelix.java:91)
        at java.lang.Thread.run(Thread.java:637)

"Low Memory Detector" daemon prio=5 tid=0x0000000103044800
nid=0x137ad5000 runnable [0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=9 tid=0x0000000103043000 nid=0x1379d2000
waiting on condition [0x0000000000000000..0x00000001379d03e0]
    java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=9 tid=0x0000000103041800 nid=0x1378cf000
waiting on condition [0x0000000000000000..0x00000001378cd3e0]
    java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=9 tid=0x0000000103041000
nid=0x1377cc000 waiting on condition
[0x0000000000000000..0x0000000000000000]
    java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x0000000103029800 nid=0x1376b2000 in
Object.wait() [0x00000001376b1000..0x00000001376b1ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115997518> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x0000000115997518> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000103028800
nid=0x1375a9000 in Object.wait()
[0x00000001375a8000..0x00000001375a8ad0]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x0000000115997190> (a java.lang.ref.Reference$Lock)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x0000000115997190> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x0000000103001000 nid=0x103901000 in Object.wait()
[0x0000000103900000..0x0000000103900c40]
    java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000001159e6a00> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1161)
        - locked <0x00000001159e6a00> (a java.lang.Thread)
        at java.lang.Thread.join(Thread.java:1214)
        at
com
.sun
.enterprise
.glassfish.bootstrap.ASMainFelix.launchOSGiFW(ASMainFelix.java:110)
        at
com.sun.enterprise.glassfish.bootstrap.ASMainOSGi.run(ASMainOSGi.java:
137)
        at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:65)

"VM Thread" prio=9 tid=0x0000000103020800 nid=0x1374a6000 runnable

"VM Periodic Task Thread" prio=10 tid=0x0000000103046000
nid=0x137bd8000 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x0000000103002000
nid=0x103a04000 runnable
JNI global references: 819


Found one Java-level deadlock:
=============================
"ComplianceMonitor.ValidatorThread":
   waiting to lock monitor 0x0000000139935b98 (object
0x000000011599a0b8, a java.util.logging.LogManager),
   which is held by "RMI TCP Connection(4)-192.168.1.5"
"RMI TCP Connection(4)-192.168.1.5":
   waiting to lock monitor 0x0000000139976ec8 (object
0x000000012b1c11b0, a java.lang.Class),
   which is held by "ComplianceMonitor.ValidatorThread"

Java stack information for the threads listed above:
===================================================
"ComplianceMonitor.ValidatorThread":
        at java.util.logging.LogManager.getLogger(LogManager.java:550)
        - waiting to lock <0x000000011599a0b8> (a java.util.logging.LogManager)
        at java.util.logging.Logger.getLogger(Logger.java:275)
        - locked <0x000000012b1c11b0> (a java.lang.Class for
java.util.logging.Logger)
        at com.sun.jmx.trace.TraceManager.getLogger(TraceManager.java:166)
        at com.sun.jmx.trace.TraceManager.isSelected(TraceManager.java:197)
        at com.sun.jmx.trace.Trace.isSelected(Trace.java:84)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.isTraceOn(DefaultMBeanServerInterceptor.java:1832)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.getAttribute(DefaultMBeanServerInterceptor.java:657)
        at
com
.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
638)
        at
org
.glassfish
.admin
.amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:251)
        at
org
.glassfish
.admin
.amx
.util.jmx.MBeanProxyHandler.getAttributeNoThrow(MBeanProxyHandler.java:
263)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.getChildren(AMXProxyHandler.java:
792)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.childrenSet(AMXProxyHandler.java:
810)
        at
org
.glassfish
.admin
.amx
.core.proxy.AMXProxyHandler.handleSpecialMethod(AMXProxyHandler.java:
421)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:594)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
        at $Proxy66.childrenSet(Unknown Source)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:226)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:234)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listChildren(PathnamesImpl.java:245)
        at
org
.glassfish
.admin.amx.impl.path.PathnamesImpl.listObjectNames(PathnamesImpl.java:
261)
        at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
1082)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:
761)
        at
javax
.management
.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:
288)
        at
org
.glassfish
.admin.amx.util.jmx.MBeanProxyHandler.invoke(MBeanProxyHandler.java:467)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:605)
        at
org
.glassfish
.admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:552)
        at $Proxy69.listObjectNames(Unknown Source)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
318)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
331)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
$ValidatorThread.doRun(ComplianceMonitor.java:134)
        at org.glassfish.admin.amx.impl.mbean.ComplianceMonitor
$ValidatorThread.run(ComplianceMonitor.java:118)
"RMI TCP Connection(4)-192.168.1.5":
        at java.util.logging.Logger.getLogger(Logger.java:274)
        - waiting to lock <0x000000012b1c11b0> (a java.lang.Class for
java.util.logging.Logger)
        at java.util.logging.LogManager.addLogger(LogManager.java:399)
        - locked <0x000000011599a0b8> (a java.util.logging.LogManager)
        at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
        at com.sun.appserv.security.AppservRealm.<clinit>(AppservRealm.java:64)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun
.reflect
.NativeConstructorAccessorImpl
.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun
.reflect
.DelegatingConstructorAccessorImpl
.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        at java.lang.Class.newInstance0(Class.java:355)
        at java.lang.Class.newInstance(Class.java:308)
        at
com.sun.enterprise.security.auth.realm.Realm.doInstantiate(Realm.java:
243)
        at
com.sun.enterprise.security.auth.realm.Realm.instantiate(Realm.java:165)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.loadRealms(RealmsMgrImpl.java:111)
        - locked <0x000000010b9c05c8> (a
org.glassfish.admin.amx.impl.ext.RealmsMgrImpl)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getRealm(RealmsMgrImpl.java:166)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getUserNames(RealmsMgrImpl.java:252)
        at
org
.glassfish
.admin.amx.impl.ext.RealmsMgrImpl.getAnonymousLogin(RealmsMgrImpl.java:
360)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org
.glassfish
.admin
.amx.impl.mbean.AMXImplBase.getAttributeByMethod(AMXImplBase.java:588)
        at
org
.glassfish
.admin
.amx.impl.mbean.AMXImplBase.getAttributeInternal(AMXImplBase.java:505)
        at
org
.glassfish
.admin.amx.impl.mbean.AMXImplBase.getAttribute(AMXImplBase.java:460)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.getAttribute(DefaultMBeanServerInterceptor.java:666)
        at
com
.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:
638)
        at
org
.glassfish
.admin
.amx.util.jmx.MBeanProxyHandler.getAttribute(MBeanProxyHandler.java:251)
        at
org.glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:
199)
        at
org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:
322)
        at
org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
157)
        at
org.glassfish.admin.amx.impl.mbean.ToolsImpl.validate(ToolsImpl.java:
169)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at
org.glassfish.admin.amx.impl.mbean.AMXImplBase.invoke(AMXImplBase.java:
1082)
        at
com
.sun
.jmx
.interceptor
.DefaultMBeanServerInterceptor
.invoke(DefaultMBeanServerInterceptor.java:836)
        at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:
761)
        at
javax
.management
.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
        at javax.management.remote.rmi.RMIConnectionImpl.access
$200(RMIConnectionImpl.java:72)
        at javax.management.remote.rmi.RMIConnectionImpl
$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
        at
javax
.management
.remote
.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:
1359)
        at
javax
.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:
788)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun
.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
        at
sun
.reflect
.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
        at sun.rmi.transport.Transport$1.run(Transport.java:159)
        at java.security.AccessController.doPrivileged(Native Method)
        at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
        at
sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
        at sun.rmi.transport.tcp.TCPTransport
$ConnectionHandler.run0(TCPTransport.java:790)
        at sun.rmi.transport.tcp.TCPTransport
$ConnectionHandler.run(TCPTransport.java:649)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.runTask(ThreadPoolExecutor.java:885)
        at java.util.concurrent.ThreadPoolExecutor
$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:637)

Found 1 deadlock.

Heap
  def new generation total 13056K, used 3194K [0x000000010aed0000,
0x000000010bd60000, 0x0000000115970000)
   eden space 11200K, 20% used [0x000000010aed0000,
0x000000010b1141d0, 0x000000010b9c0000)
   from space 1856K, 47% used [0x000000010b9c0000,
0x000000010ba9a7f8, 0x000000010bb90000)
   to space 1856K, 0% used [0x000000010bb90000,
0x000000010bb90000, 0x000000010bd60000)
  tenured generation total 29580K, used 23266K [0x0000000115970000,
0x0000000117653000, 0x000000012aed0000)
    the space 29580K, 78% used [0x0000000115970000,
0x0000000117028970, 0x0000000117028a00, 0x0000000117653000)
  compacting perm gen total 25408K, used 25108K [0x000000012aed0000,
0x000000012c7a0000, 0x0000000136ed0000)
    the space 25408K, 98% used [0x000000012aed0000,
0x000000012c7552d0, 0x000000012c755400, 0x000000012c7a0000)
No shared spaces configured.