dev@glassfish.java.net

Re: v3 build hanging

From: Hong Zhang <Hong.Zhang_at_Sun.COM>
Date: Tue, 14 Jul 2009 12:51:22 -0400

The stack trace I attached earlier was for creating-doman, and I have
now attached the stack trace for starting the domain which is probably
what we need to look at.

In the meantime, any workaround for me to go further?


Hong Zhang wrote:

>
>>> Since I'm not able to reproduce this hang. Is it possible if you
>>> can execute "jstack -l pid" to produce the stack dump?
>>>
>> I have just cleaned my local repository to see if I can get a
>> successful build as I want to move forward to do other work. If I get
>> this error again, I will try jstack -l pid.
>
>
> Ok, it still hangs for me after I started with a clean repository:
> hzhang_at_nmr:~/.m2$ jstack -l 10965
> 2009-07-14 12:35:30
> Full thread dump Java HotSpot(TM) Server VM (11.0-b16 mixed mode):
>
> "Attach Listener" daemon prio=10 tid=0x081b1800 nid=0x2b6c waiting on
> condition [0x00000000..0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "GlassFish Kernel Main Thread" prio=10 tid=0x7c226c00 nid=0x2b1f in
> Object.wait() [0x7c1ad000..0x7c1ade30]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xafa12e60> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
> at java.lang.Object.wait(Object.java:485)
> at
> com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:144)
>
> - locked <0xafa12e60> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
>
> Locked ownable synchronizers:
> - None
>
> "AutoDeployer" daemon prio=10 tid=0x083a3c00 nid=0x2b1e in
> Object.wait() [0x7c1fe000..0x7c1fedb0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xaf8501d8> (a java.util.TaskQueue)
> at java.lang.Object.wait(Object.java:485)
> at java.util.TimerThread.mainLoop(Timer.java:483)
> - locked <0xaf8501d8> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> Locked ownable synchronizers:
> - None
>
> "DynamicReloader" daemon prio=10 tid=0x083a4800 nid=0x2b1d in
> Object.wait() [0x7c451000..0x7c451f30]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xaf741b70> (a java.util.TaskQueue)
> at java.lang.Object.wait(Object.java:485)
> at java.util.TimerThread.mainLoop(Timer.java:483)
> - locked <0xaf741b70> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> Locked ownable synchronizers:
> - None
>
> "pool-2-thread-1" daemon prio=10 tid=0x7c21e000 nid=0x2b1b waiting on
> condition [0x7c4a2000..0x7c4a3030]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xaf2293a8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:619)
>
> Locked ownable synchronizers:
> - <0xaf229490> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
> "pool-1-thread-1" daemon prio=10 tid=0x7c216800 nid=0x2b1a waiting on
> condition [0x7c4f3000..0x7c4f3fb0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0xaf1c5968> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
> at
> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>
> at java.lang.Thread.run(Thread.java:619)
>
> Locked ownable synchronizers:
> - <0xaf1ca4c8> (a
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>
> "Low Memory Detector" daemon prio=10 tid=0x080f9400 nid=0x2ae3
> runnable [0x00000000..0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "CompilerThread1" daemon prio=10 tid=0x080f7400 nid=0x2ae2 waiting on
> condition [0x00000000..0x7c984428]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "CompilerThread0" daemon prio=10 tid=0x080f4400 nid=0x2ae1 waiting on
> condition [0x00000000..0x7ca05468]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Signal Dispatcher" daemon prio=10 tid=0x080f2c00 nid=0x2ae0 runnable
> [0x00000000..0x7ca56e10]
> java.lang.Thread.State: RUNNABLE
>
> Locked ownable synchronizers:
> - None
>
> "Finalizer" daemon prio=10 tid=0x080dec00 nid=0x2adf in Object.wait()
> [0x7cae9000..0x7caea0b0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> Locked ownable synchronizers:
> - None
>
> "Reference Handler" daemon prio=10 tid=0x080dd400 nid=0x2ade in
> Object.wait() [0x7cb3a000..0x7cb3ae30]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xb021d900> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at
> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0xb021d900> (a java.lang.ref.Reference$Lock)
>
> Locked ownable synchronizers:
> - None
>
> "main" prio=10 tid=0x08058800 nid=0x2ada in Object.wait()
> [0xb7d71000..0xb7d721f8]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0xafa12e60> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
> at java.lang.Thread.join(Thread.java:1143)
> - locked <0xafa12e60> (a
> com.sun.enterprise.v3.server.AppServerStartup$1)
> at
> com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:348)
>
> at org.glassfish.api.embedded.Server.stop(Server.java:359)
> at
> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.modifyInitialDomainXml(CreateDomainCommand.java:773)
>
> at
> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.createTheDomain(CreateDomainCommand.java:496)
>
> at
> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.runCommand(CreateDomainCommand.java:267)
>
> at
> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:171)
> at
> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:105)
> at
> com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:190)
> at
> com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:84)
>
> Locked ownable synchronizers:
> - None
>
> "VM Thread" prio=10 tid=0x080d9c00 nid=0x2add runnable
>
> "GC task thread#0 (ParallelGC)" prio=10 tid=0x08060000 nid=0x2adb
> runnable
>
> "GC task thread#1 (ParallelGC)" prio=10 tid=0x08061400 nid=0x2adc
> runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x080fb000 nid=0x2ae4 waiting on
> condition
>
> JNI global references: 842
>
>
>>
>> Thanks,
>>
>> - Hong
>>
>>>
>>>
>>> Hong Zhang wrote:
>>>
>>>>
>>>>> I just built successfully on the Mac using an existing Maven repo.
>>>>> Looks like this hang is happening sporadically.
>>>>> BTW: are you using a Linux? I recall seeing the same hang in
>>>>> create-domain on the Hudson dev job yesterday. Hudson dev job is
>>>>> also running on Linux.
>>>>>
>>>> Yes, I am on ubuntu.
>>>>
>>>>>
>>>>> Hong Zhang wrote:
>>>>>
>>>>>>
>>>>>>>>> in which module is this ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Most times in core/kernel. There was one time the build was
>>>>>>>> able to go further, and then when it's building the nucleus
>>>>>>>> distribution, it hang with a similar error.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ok let me know if it hangs somewhere else. I need to spawn a
>>>>>>> process to test the embedded as running several embedded
>>>>>>> instances in the same process can have side effects.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Jerome: I have updated workspace and got your changes. Now it
>>>>>> hang in building nucleus distribution:
>>>>>>
>>>>>> create.distribution:
>>>>>> [echo] Creating distribution: nucleus
>>>>>> [unzip] Expanding:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/felix-3.0-SNAPSHOT.zip
>>>>>> into
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>> [unzip] Expanding:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-grizzly-3.0-SNAPSHOT.zip
>>>>>> into
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>> [unzip] Expanding:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-hk2-3.0-SNAPSHOT.zip
>>>>>> into
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>> [unzip] Expanding:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-nucleus-3.0-SNAPSHOT.zip
>>>>>> into
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>> [mkdir] Created dir:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/target
>>>>>> [delete] DEPRECATED - Use of the implicit FileSet is
>>>>>> deprecated. Use a nested fileset element instead.
>>>>>> [delete] Deleting 4 files from
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>
>>>>>> fetch-domain-creation-artifacts:
>>>>>> [echo] cli-optional jar:
>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/cli-optional/3.0-SNAPSHOT/cli-optional-3.0-SNAPSHOT.jar
>>>>>>
>>>>>> [echo] server-mgmt jar:
>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/server-mgmt/3.0-SNAPSHOT/server-mgmt-3.0-SNAPSHOT.jar
>>>>>>
>>>>>> [echo] admin-util jar:
>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/admin-util/3.0-SNAPSHOT/admin-util-3.0-SNAPSHOT.jar
>>>>>>
>>>>>> [echo] security jar:
>>>>>> /home/hzhang/.m2/repository/org/glassfish/security/security/3.0-SNAPSHOT/security-3.0-SNAPSHOT.jar
>>>>>>
>>>>>> [copy] Copying 1 file to
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>
>>>>>> [copy] Copying 1 file to
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>
>>>>>> [copy] Copying 1 file to
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>
>>>>>> [copy] Copying 1 file to
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>
>>>>>>
>>>>>> init:
>>>>>>
>>>>>> create-domain-on-windows:
>>>>>>
>>>>>> create-domain-on-unix:
>>>>>>
>>>>>> create-domain:
>>>>>> [echo] Directory to run command from:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin
>>>>>>
>>>>>> [echo] Command to run:
>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin/asadmin
>>>>>>
>>>>>> [exec] Using port 4848 for Admin.
>>>>>> [exec] Using port 8080 for HTTP Instance.
>>>>>> [exec] Using default port 7676 for JMS.
>>>>>> [exec] Using default port 3700 for IIOP.
>>>>>> [exec] Using default port 8181 for HTTP_SSL.
>>>>>> [exec] Using default port 3820 for IIOP_SSL.
>>>>>> [exec] Using default port 3920 for IIOP_MUTUALAUTH.
>>>>>> [exec] Using default port 8686 for JMX_ADMIN.
>>>>>> [exec] Distinguished Name of the self-signed X.509 Server
>>>>>> Certificate is:
>>>>>> [exec] [CN=localhost,OU=GlassFish,O=Sun Microsystems,L=Santa
>>>>>> Clara,ST=California,C=US]
>>>>>> [exec] Jul 14, 2009 11:51:55 AM
>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>> [exec] INFO: GlassFish v3 startup time : Embedded(307ms)
>>>>>> startup services(224ms) total(531ms)
>>>>>> [exec] Jul 14, 2009 11:51:55 AM AppServerStartup run
>>>>>> [exec] INFO: [Thread[GlassFish Kernel Main Thread,5,main]]
>>>>>> started
>>>>>> [exec] Received Stop
>>>>>> [exec] Sending stop event to ms
>>>>>>
>>>>>>
>>>>>> - Hong
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> you updated and did mvn clean install from the root ?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, I updated, and did mvn -U clean install from the root.
>>>>>>>>
>>>>>>>> - Hong
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Jul 14, 2009, at 8:10 AM, Hong Zhang wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I have updated my workspace this morning and tried to build
>>>>>>>>>> the workspace and the build kept hanging. I have restarted
>>>>>>>>>> a few times and no difference. There is where it hangs:
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>> T E S T S
>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>> Running com.sun.enterprise.v3.admin.AdminAdapterTest
>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
>>>>>>>>>> elapsed: 0.019 sec
>>>>>>>>>> Running com.sun.enterprise.v3.admin.CreateProfilerTest
>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time
>>>>>>>>>> elapsed: 0.001 sec
>>>>>>>>>> Running org.glassfish.tests.kernel.embedded.EmbeddedTest
>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>>>>>> INFO: GlassFish v3 startup time : Embedded(219ms) startup
>>>>>>>>>> services(170ms) total(389ms)
>>>>>>>>>> Jul 14, 2009 10:55:43 AM AppServerStartup run
>>>>>>>>>> INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>> com.sun.enterprise.v3.admin.CommandRunnerImpl getCommand
>>>>>>>>>> INFO: Command create-http-listener not found
>>>>>>>>>> Received Stop
>>>>>>>>>> Sending stop event to ms
>>>>>>>>>>
>>>>>>>>>> Anyone seeing similar thing? I have svn revision 29132.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> - Hong
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>





2009-07-14 12:46:11
Full thread dump Java HotSpot(TM) Client VM (11.0-b16 mixed mode, sharing):

"Attach Listener" daemon prio=10 tid=0x08255c00 nid=0x2bfb waiting on condition [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"{felix.fileinstall.poll=5000, felix.fileinstall.bundles.new.start=true, felix.fileinstall.dir=/home/hzhang/files/sun/glassfishv3/glassfish/domains/domain1/autodeploy-bundles/, felix.fileinstall.debug=1}" daemon prio=10 tid=0xb3cb1800 nid=0x29ae waiting on condition [0xb3dfe000..0xb3dfef30]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
        at java.lang.Thread.sleep(Native Method)
        at org.apache.felix.fileinstall.DirectoryWatcher.run(DirectoryWatcher.java:125)

   Locked ownable synchronizers:
        - None

"GlassFish Kernel Main Thread" prio=10 tid=0xb3cc9000 nid=0x29ad in Object.wait() [0xb3bfe000..0xb3bfeeb0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x73b13be8> (a com.sun.enterprise.v3.server.AppServerStartup$1)
        at java.lang.Object.wait(Object.java:485)
        at com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:142)
        - locked <0x73b13be8> (a com.sun.enterprise.v3.server.AppServerStartup$1)

   Locked ownable synchronizers:
        - None

"Thread-10" daemon prio=10 tid=0x082a5000 nid=0x29a6 runnable [0xb39b7000..0xb39b7e30]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0x738a9ce8> (a sun.nio.ch.Util$1)
        - locked <0x738a9cf8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x738a9ca8> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:492)
        at com.sun.grizzly.Controller.doSelect(Controller.java:329)
        at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:88)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x737f6910> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-11" daemon prio=10 tid=0x082a3c00 nid=0x29a5 runnable [0xb3a08000..0xb3a08db0]
   java.lang.Thread.State: RUNNABLE
        at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
        at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        - locked <0x737ec1b8> (a sun.nio.ch.Util$1)
        - locked <0x737ec1c8> (a java.util.Collections$UnmodifiableSet)
        - locked <0x737ec178> (a sun.nio.ch.EPollSelectorImpl)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at com.sun.grizzly.TCPSelectorHandler.select(TCPSelectorHandler.java:492)
        at com.sun.grizzly.Controller.doSelect(Controller.java:329)
        at com.sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:88)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x737f6a58> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"AutoDeployer" daemon prio=10 tid=0x08299c00 nid=0x29a4 in Object.wait() [0xb3a59000..0xb3a59f30]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0x737ec330> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

   Locked ownable synchronizers:
        - None

"DynamicReloader" daemon prio=10 tid=0xb40f7400 nid=0x29a3 in Object.wait() [0xb3f5c000..0xb3f5ceb0]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.util.TimerThread.mainLoop(Timer.java:509)
        - locked <0x73629448> (a java.util.TaskQueue)
        at java.util.TimerThread.run(Timer.java:462)

   Locked ownable synchronizers:
        - None

"Thread-5" daemon prio=10 tid=0x08294000 nid=0x29a2 in Object.wait() [0xb3aaa000..0xb3aab030]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1211)
        - locked <0x737ec3a0> (a java.util.concurrent.atomic.AtomicInteger)
        at com.sun.grizzly.Controller.start(Controller.java:968)
        at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1191)
        - locked <0x7363aab0> (a java.lang.Object)
        at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1039)
        at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1118)
        at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:118)
        at com.sun.enterprise.v3.services.impl.GrizzlyProxy$1.run(GrizzlyProxy.java:218)

   Locked ownable synchronizers:
        - None

"Thread-4" daemon prio=10 tid=0x08289800 nid=0x29a1 in Object.wait() [0xb3afb000..0xb3afbfb0]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at com.sun.grizzly.Controller.waitUntilSeletorHandlersStop(Controller.java:1211)
        - locked <0x737ec3b0> (a java.util.concurrent.atomic.AtomicInteger)
        at com.sun.grizzly.Controller.start(Controller.java:968)
        at com.sun.grizzly.http.SelectorThread.startListener(SelectorThread.java:1191)
        - locked <0x7363ae88> (a java.lang.Object)
        at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1039)
        at com.sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:1118)
        at com.sun.enterprise.v3.services.impl.GrizzlyListener.startEndpoint(GrizzlyListener.java:118)
        at com.sun.enterprise.v3.services.impl.GrizzlyProxy$1.run(GrizzlyProxy.java:218)

   Locked ownable synchronizers:
        - None

"pool-7-thread-1" daemon prio=10 tid=0xb40f8c00 nid=0x299f waiting on condition [0xb3b4c000..0xb3b4d0b0]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x736298c8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x7363b230> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Thread-1" daemon prio=10 tid=0xb4ee5000 nid=0x299c waiting on condition [0xb3fad000..0xb3fadf30]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x7334f828> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at com.sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:498)
        at com.sun.enterprise.server.logging.GFFileHandler$3.run(GFFileHandler.java:246)

   Locked ownable synchronizers:
        - None

"pool-4-thread-1" daemon prio=10 tid=0xb41f5800 nid=0x299b waiting on condition [0xb3ffe000..0xb3ffeeb0]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x7334f968> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
        at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - None

"pool-3-thread-1" daemon prio=10 tid=0xb4ee8c00 nid=0x299a waiting on condition [0xb427e000..0xb427f030]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x732bb298> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x732d1c88> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"pool-2-thread-1" daemon prio=10 tid=0xb4edfc00 nid=0x2999 waiting on condition [0xb42cf000..0xb42cffb0]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x732ddbe8> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x732d1d08> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"pool-1-thread-1" daemon prio=10 tid=0xb41ee000 nid=0x2998 waiting on condition [0xb4460000..0xb4461130]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for <0x7324e428> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
        at java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
        at org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - <0x73266cf0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)

"Configuration Updater" daemon prio=10 tid=0xb50ab000 nid=0x2997 in Object.wait() [0xb4fad000..0xb4fae0b0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x730d4e48> (a java.util.LinkedList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
        - locked <0x730d4e48> (a java.util.LinkedList)

   Locked ownable synchronizers:
        - None

"FelixPackageAdmin" daemon prio=10 tid=0xb5091400 nid=0x2995 in Object.wait() [0xb4ffe000..0xb4ffedb0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x72b740a0> (a org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:314)
        - locked <0x72b740a0> (a org.apache.felix.framework.PackageAdminImpl)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - None

"FelixStartLevel" daemon prio=10 tid=0xb5092c00 nid=0x2994 in Object.wait() [0xb514f000..0xb514ff30]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x72b741c8> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:241)
        - locked <0x72b741c8> (a java.util.ArrayList)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - None

"FelixDispatchQueue" prio=10 tid=0xb5094000 nid=0x2993 in Object.wait() [0xb51a0000..0xb51a0eb0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x72b74248> (a java.util.ArrayList)
        at java.lang.Object.wait(Object.java:485)
        at org.apache.felix.framework.util.EventDispatcher.run(EventDispatcher.java:875)
        - locked <0x72b74248> (a java.util.ArrayList)
        at org.apache.felix.framework.util.EventDispatcher.access$000(EventDispatcher.java:50)
        at org.apache.felix.framework.util.EventDispatcher$1.run(EventDispatcher.java:102)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - None

"DestroyJavaVM" prio=10 tid=0xb55e7c00 nid=0x2988 waiting on condition [0x00000000..0xb7dcb070]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"OSGi Framework Launcher" prio=10 tid=0xb55e6000 nid=0x2990 in Object.wait() [0xb536e000..0xb536f030]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x72b76b08> (a org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:48)
        - locked <0x72b76b08> (a org.apache.felix.framework.util.ThreadGate)
        at org.apache.felix.framework.Felix.waitForStop(Felix.java:785)
        at org.apache.felix.main.Main.main(Main.java:215)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.sun.enterprise.glassfish.bootstrap.ASMainFelix$1.run(ASMainFelix.java:102)
        at java.lang.Thread.run(Thread.java:619)

   Locked ownable synchronizers:
        - None

"Low Memory Detector" daemon prio=10 tid=0x08092400 nid=0x298e runnable [0x00000000..0x00000000]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"CompilerThread0" daemon prio=10 tid=0x0808f000 nid=0x298d waiting on condition [0x00000000..0xb5835b68]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Signal Dispatcher" daemon prio=10 tid=0x0808d800 nid=0x298c runnable [0x00000000..0xb5886b10]
   java.lang.Thread.State: RUNNABLE

   Locked ownable synchronizers:
        - None

"Finalizer" daemon prio=10 tid=0x08084000 nid=0x298b in Object.wait() [0xb5919000..0xb5919db0]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x72b2bba8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
        - locked <0x72b2bba8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
        at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

   Locked ownable synchronizers:
        - None

"Reference Handler" daemon prio=10 tid=0x08082c00 nid=0x298a in Object.wait() [0xb596a000..0xb596af30]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x72b2ba00> (a java.lang.ref.Reference$Lock)

   Locked ownable synchronizers:
        - None

"VM Thread" prio=10 tid=0x08081000 nid=0x2989 runnable

"VM Periodic Task Thread" prio=10 tid=0x08094000 nid=0x298f waiting on condition

JNI global references: 983