Thanks for the stacktrace. As Jerome pointed out in v3 mtg, this maybe
related to the change in CreateDomainCommand.java, rev 29043
Can you checkout the source from rev 29041: "svn checkout -r 29041
https://svn.dev.java.net/svn/glassfish-svn/trunk/v3" and do a build?
Please let me know if the build succeeds.
Thanks,
Jane
Hong Zhang wrote:
> The stack trace I attached earlier was for creating-doman, and I have
> now attached the stack trace for starting the domain which is probably
> what we need to look at.
>
> In the meantime, any workaround for me to go further?
>
>
> Hong Zhang wrote:
>>
>>>> Since I'm not able to reproduce this hang. Is it possible if you
>>>> can execute "jstack -l pid" to produce the stack dump?
>>>>
>>> I have just cleaned my local repository to see if I can get a
>>> successful build as I want to move forward to do other work. If I
>>> get this error again, I will try jstack -l pid.
>>
>> Ok, it still hangs for me after I started with a clean repository:
>> hzhang_at_nmr:~/.m2$ jstack -l 10965
>> 2009-07-14 12:35:30
>> Full thread dump Java HotSpot(TM) Server VM (11.0-b16 mixed mode):
>>
>> "Attach Listener" daemon prio=10 tid=0x081b1800 nid=0x2b6c waiting on
>> condition [0x00000000..0x00000000]
>> java.lang.Thread.State: RUNNABLE
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "GlassFish Kernel Main Thread" prio=10 tid=0x7c226c00 nid=0x2b1f in
>> Object.wait() [0x7c1ad000..0x7c1ade30]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xafa12e60> (a
>> com.sun.enterprise.v3.server.AppServerStartup$1)
>> at java.lang.Object.wait(Object.java:485)
>> at
>> com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:144)
>>
>> - locked <0xafa12e60> (a
>> com.sun.enterprise.v3.server.AppServerStartup$1)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "AutoDeployer" daemon prio=10 tid=0x083a3c00 nid=0x2b1e in
>> Object.wait() [0x7c1fe000..0x7c1fedb0]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xaf8501d8> (a java.util.TaskQueue)
>> at java.lang.Object.wait(Object.java:485)
>> at java.util.TimerThread.mainLoop(Timer.java:483)
>> - locked <0xaf8501d8> (a java.util.TaskQueue)
>> at java.util.TimerThread.run(Timer.java:462)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "DynamicReloader" daemon prio=10 tid=0x083a4800 nid=0x2b1d in
>> Object.wait() [0x7c451000..0x7c451f30]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xaf741b70> (a java.util.TaskQueue)
>> at java.lang.Object.wait(Object.java:485)
>> at java.util.TimerThread.mainLoop(Timer.java:483)
>> - locked <0xaf741b70> (a java.util.TaskQueue)
>> at java.util.TimerThread.run(Timer.java:462)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "pool-2-thread-1" daemon prio=10 tid=0x7c21e000 nid=0x2b1b waiting on
>> condition [0x7c4a2000..0x7c4a3030]
>> java.lang.Thread.State: WAITING (parking)
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for <0xaf2293a8> (a
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>> at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>>
>> at
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>>
>> at
>> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>
>> at java.lang.Thread.run(Thread.java:619)
>>
>> Locked ownable synchronizers:
>> - <0xaf229490> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>
>> "pool-1-thread-1" daemon prio=10 tid=0x7c216800 nid=0x2b1a waiting on
>> condition [0x7c4f3000..0x7c4f3fb0]
>> java.lang.Thread.State: WAITING (parking)
>> at sun.misc.Unsafe.park(Native Method)
>> - parking to wait for <0xaf1c5968> (a
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>> at
>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>> at
>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>>
>> at
>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>>
>> at
>> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
>> at
>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>
>> at java.lang.Thread.run(Thread.java:619)
>>
>> Locked ownable synchronizers:
>> - <0xaf1ca4c8> (a
>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>
>> "Low Memory Detector" daemon prio=10 tid=0x080f9400 nid=0x2ae3
>> runnable [0x00000000..0x00000000]
>> java.lang.Thread.State: RUNNABLE
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "CompilerThread1" daemon prio=10 tid=0x080f7400 nid=0x2ae2 waiting on
>> condition [0x00000000..0x7c984428]
>> java.lang.Thread.State: RUNNABLE
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "CompilerThread0" daemon prio=10 tid=0x080f4400 nid=0x2ae1 waiting on
>> condition [0x00000000..0x7ca05468]
>> java.lang.Thread.State: RUNNABLE
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "Signal Dispatcher" daemon prio=10 tid=0x080f2c00 nid=0x2ae0 runnable
>> [0x00000000..0x7ca56e10]
>> java.lang.Thread.State: RUNNABLE
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "Finalizer" daemon prio=10 tid=0x080dec00 nid=0x2adf in Object.wait()
>> [0x7cae9000..0x7caea0b0]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
>> - locked <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
>> at
>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "Reference Handler" daemon prio=10 tid=0x080dd400 nid=0x2ade in
>> Object.wait() [0x7cb3a000..0x7cb3ae30]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xb021d900> (a java.lang.ref.Reference$Lock)
>> at java.lang.Object.wait(Object.java:485)
>> at
>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
>> - locked <0xb021d900> (a java.lang.ref.Reference$Lock)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "main" prio=10 tid=0x08058800 nid=0x2ada in Object.wait()
>> [0xb7d71000..0xb7d721f8]
>> java.lang.Thread.State: WAITING (on object monitor)
>> at java.lang.Object.wait(Native Method)
>> - waiting on <0xafa12e60> (a
>> com.sun.enterprise.v3.server.AppServerStartup$1)
>> at java.lang.Thread.join(Thread.java:1143)
>> - locked <0xafa12e60> (a
>> com.sun.enterprise.v3.server.AppServerStartup$1)
>> at
>> com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:348)
>>
>> at org.glassfish.api.embedded.Server.stop(Server.java:359)
>> at
>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.modifyInitialDomainXml(CreateDomainCommand.java:773)
>>
>> at
>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.createTheDomain(CreateDomainCommand.java:496)
>>
>> at
>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.runCommand(CreateDomainCommand.java:267)
>>
>> at
>> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:171)
>> at
>> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:105)
>> at
>> com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:190)
>> at
>> com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:84)
>>
>> Locked ownable synchronizers:
>> - None
>>
>> "VM Thread" prio=10 tid=0x080d9c00 nid=0x2add runnable
>>
>> "GC task thread#0 (ParallelGC)" prio=10 tid=0x08060000 nid=0x2adb
>> runnable
>>
>> "GC task thread#1 (ParallelGC)" prio=10 tid=0x08061400 nid=0x2adc
>> runnable
>>
>> "VM Periodic Task Thread" prio=10 tid=0x080fb000 nid=0x2ae4 waiting
>> on condition
>>
>> JNI global references: 842
>>
>>
>>>
>>> Thanks,
>>>
>>> - Hong
>>>
>>>>
>>>>
>>>> Hong Zhang wrote:
>>>>
>>>>>
>>>>>> I just built successfully on the Mac using an existing Maven repo.
>>>>>> Looks like this hang is happening sporadically.
>>>>>> BTW: are you using a Linux? I recall seeing the same hang in
>>>>>> create-domain on the Hudson dev job yesterday. Hudson dev job is
>>>>>> also running on Linux.
>>>>>>
>>>>> Yes, I am on ubuntu.
>>>>>
>>>>>>
>>>>>> Hong Zhang wrote:
>>>>>>
>>>>>>>
>>>>>>>>>> in which module is this ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Most times in core/kernel. There was one time the build was
>>>>>>>>> able to go further, and then when it's building the nucleus
>>>>>>>>> distribution, it hang with a similar error.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ok let me know if it hangs somewhere else. I need to spawn a
>>>>>>>> process to test the embedded as running several embedded
>>>>>>>> instances in the same process can have side effects.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Jerome: I have updated workspace and got your changes. Now it
>>>>>>> hang in building nucleus distribution:
>>>>>>>
>>>>>>> create.distribution:
>>>>>>> [echo] Creating distribution: nucleus
>>>>>>> [unzip] Expanding:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/felix-3.0-SNAPSHOT.zip
>>>>>>> into
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>> [unzip] Expanding:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-grizzly-3.0-SNAPSHOT.zip
>>>>>>> into
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>> [unzip] Expanding:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-hk2-3.0-SNAPSHOT.zip
>>>>>>> into
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>> [unzip] Expanding:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-nucleus-3.0-SNAPSHOT.zip
>>>>>>> into
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>> [mkdir] Created dir:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/target
>>>>>>> [delete] DEPRECATED - Use of the implicit FileSet is
>>>>>>> deprecated. Use a nested fileset element instead.
>>>>>>> [delete] Deleting 4 files from
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>
>>>>>>> fetch-domain-creation-artifacts:
>>>>>>> [echo] cli-optional jar:
>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/cli-optional/3.0-SNAPSHOT/cli-optional-3.0-SNAPSHOT.jar
>>>>>>>
>>>>>>> [echo] server-mgmt jar:
>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/server-mgmt/3.0-SNAPSHOT/server-mgmt-3.0-SNAPSHOT.jar
>>>>>>>
>>>>>>> [echo] admin-util jar:
>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/admin-util/3.0-SNAPSHOT/admin-util-3.0-SNAPSHOT.jar
>>>>>>>
>>>>>>> [echo] security jar:
>>>>>>> /home/hzhang/.m2/repository/org/glassfish/security/security/3.0-SNAPSHOT/security-3.0-SNAPSHOT.jar
>>>>>>>
>>>>>>> [copy] Copying 1 file to
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>
>>>>>>> [copy] Copying 1 file to
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>
>>>>>>> [copy] Copying 1 file to
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>
>>>>>>> [copy] Copying 1 file to
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>
>>>>>>>
>>>>>>> init:
>>>>>>>
>>>>>>> create-domain-on-windows:
>>>>>>>
>>>>>>> create-domain-on-unix:
>>>>>>>
>>>>>>> create-domain:
>>>>>>> [echo] Directory to run command from:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin
>>>>>>>
>>>>>>> [echo] Command to run:
>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin/asadmin
>>>>>>>
>>>>>>> [exec] Using port 4848 for Admin.
>>>>>>> [exec] Using port 8080 for HTTP Instance.
>>>>>>> [exec] Using default port 7676 for JMS.
>>>>>>> [exec] Using default port 3700 for IIOP.
>>>>>>> [exec] Using default port 8181 for HTTP_SSL.
>>>>>>> [exec] Using default port 3820 for IIOP_SSL.
>>>>>>> [exec] Using default port 3920 for IIOP_MUTUALAUTH.
>>>>>>> [exec] Using default port 8686 for JMX_ADMIN.
>>>>>>> [exec] Distinguished Name of the self-signed X.509 Server
>>>>>>> Certificate is:
>>>>>>> [exec] [CN=localhost,OU=GlassFish,O=Sun Microsystems,L=Santa
>>>>>>> Clara,ST=California,C=US]
>>>>>>> [exec] Jul 14, 2009 11:51:55 AM
>>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>>> [exec] INFO: GlassFish v3 startup time : Embedded(307ms)
>>>>>>> startup services(224ms) total(531ms)
>>>>>>> [exec] Jul 14, 2009 11:51:55 AM AppServerStartup run
>>>>>>> [exec] INFO: [Thread[GlassFish Kernel Main Thread,5,main]]
>>>>>>> started
>>>>>>> [exec] Received Stop
>>>>>>> [exec] Sending stop event to ms
>>>>>>>
>>>>>>>
>>>>>>> - Hong
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> you updated and did mvn clean install from the root ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes, I updated, and did mvn -U clean install from the root.
>>>>>>>>>
>>>>>>>>> - Hong
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Jul 14, 2009, at 8:10 AM, Hong Zhang wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I have updated my workspace this morning and tried to build
>>>>>>>>>>> the workspace and the build kept hanging. I have restarted
>>>>>>>>>>> a few times and no difference. There is where it hangs:
>>>>>>>>>>>
>>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>>> T E S T S
>>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>>> Running com.sun.enterprise.v3.admin.AdminAdapterTest
>>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
>>>>>>>>>>> elapsed: 0.019 sec
>>>>>>>>>>> Running com.sun.enterprise.v3.admin.CreateProfilerTest
>>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time
>>>>>>>>>>> elapsed: 0.001 sec
>>>>>>>>>>> Running org.glassfish.tests.kernel.embedded.EmbeddedTest
>>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>>>>>>> INFO: GlassFish v3 startup time : Embedded(219ms) startup
>>>>>>>>>>> services(170ms) total(389ms)
>>>>>>>>>>> Jul 14, 2009 10:55:43 AM AppServerStartup run
>>>>>>>>>>> INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
>>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>>> com.sun.enterprise.v3.admin.CommandRunnerImpl getCommand
>>>>>>>>>>> INFO: Command create-http-listener not found
>>>>>>>>>>> Received Stop
>>>>>>>>>>> Sending stop event to ms
>>>>>>>>>>>
>>>>>>>>>>> Anyone seeing similar thing? I have svn revision 29132.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>>
>>>>>>>>>>> - Hong
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
>
> ------------------------------------------------------------------------
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net