dev@glassfish.java.net

Re: v3 build hanging

From: Hong Zhang <Hong.Zhang_at_Sun.COM>
Date: Tue, 14 Jul 2009 15:03:17 -0400

Hi, Jane

> Thanks for the stacktrace. As Jerome pointed out in v3 mtg, this
> maybe related to the change in CreateDomainCommand.java, rev 29043
>
I followed Nandini's suggestion to comment out modifyInitialDomainXML
method on CreateDomainCommand.java (line number 496) and I was able to
build afterwards.

Thanks,

- Hong

> Can you checkout the source from rev 29041: "svn checkout -r 29041
> https://svn.dev.java.net/svn/glassfish-svn/trunk/v3" and do a build?
> Please let me know if the build succeeds.
>
> Thanks,
> Jane
>
>
> Hong Zhang wrote:
>
>> The stack trace I attached earlier was for creating-doman, and I
>> have now attached the stack trace for starting the domain which is
>> probably what we need to look at.
>>
>> In the meantime, any workaround for me to go further?
>>
>>
>> Hong Zhang wrote:
>>
>>>
>>>>> Since I'm not able to reproduce this hang. Is it possible if you
>>>>> can execute "jstack -l pid" to produce the stack dump?
>>>>>
>>>> I have just cleaned my local repository to see if I can get a
>>>> successful build as I want to move forward to do other work. If I
>>>> get this error again, I will try jstack -l pid.
>>>
>>>
>>> Ok, it still hangs for me after I started with a clean repository:
>>> hzhang_at_nmr:~/.m2$ jstack -l 10965
>>> 2009-07-14 12:35:30
>>> Full thread dump Java HotSpot(TM) Server VM (11.0-b16 mixed mode):
>>>
>>> "Attach Listener" daemon prio=10 tid=0x081b1800 nid=0x2b6c waiting
>>> on condition [0x00000000..0x00000000]
>>> java.lang.Thread.State: RUNNABLE
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "GlassFish Kernel Main Thread" prio=10 tid=0x7c226c00 nid=0x2b1f in
>>> Object.wait() [0x7c1ad000..0x7c1ade30]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xafa12e60> (a
>>> com.sun.enterprise.v3.server.AppServerStartup$1)
>>> at java.lang.Object.wait(Object.java:485)
>>> at
>>> com.sun.enterprise.v3.server.AppServerStartup$1.run(AppServerStartup.java:144)
>>>
>>> - locked <0xafa12e60> (a
>>> com.sun.enterprise.v3.server.AppServerStartup$1)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "AutoDeployer" daemon prio=10 tid=0x083a3c00 nid=0x2b1e in
>>> Object.wait() [0x7c1fe000..0x7c1fedb0]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xaf8501d8> (a java.util.TaskQueue)
>>> at java.lang.Object.wait(Object.java:485)
>>> at java.util.TimerThread.mainLoop(Timer.java:483)
>>> - locked <0xaf8501d8> (a java.util.TaskQueue)
>>> at java.util.TimerThread.run(Timer.java:462)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "DynamicReloader" daemon prio=10 tid=0x083a4800 nid=0x2b1d in
>>> Object.wait() [0x7c451000..0x7c451f30]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xaf741b70> (a java.util.TaskQueue)
>>> at java.lang.Object.wait(Object.java:485)
>>> at java.util.TimerThread.mainLoop(Timer.java:483)
>>> - locked <0xaf741b70> (a java.util.TaskQueue)
>>> at java.util.TimerThread.run(Timer.java:462)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "pool-2-thread-1" daemon prio=10 tid=0x7c21e000 nid=0x2b1b waiting
>>> on condition [0x7c4a2000..0x7c4a3030]
>>> java.lang.Thread.State: WAITING (parking)
>>> at sun.misc.Unsafe.park(Native Method)
>>> - parking to wait for <0xaf2293a8> (a
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>>> at
>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>> at
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>>>
>>> at
>>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>>>
>>> at
>>> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>
>>> at java.lang.Thread.run(Thread.java:619)
>>>
>>> Locked ownable synchronizers:
>>> - <0xaf229490> (a
>>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>>
>>> "pool-1-thread-1" daemon prio=10 tid=0x7c216800 nid=0x2b1a waiting
>>> on condition [0x7c4f3000..0x7c4f3fb0]
>>> java.lang.Thread.State: WAITING (parking)
>>> at sun.misc.Unsafe.park(Native Method)
>>> - parking to wait for <0xaf1c5968> (a
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
>>> at
>>> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
>>> at
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
>>>
>>> at
>>> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:317)
>>>
>>> at
>>> org.jvnet.hk2.config.Transactions$Notifier$1.run(Transactions.java:142)
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>
>>> at java.lang.Thread.run(Thread.java:619)
>>>
>>> Locked ownable synchronizers:
>>> - <0xaf1ca4c8> (a
>>> java.util.concurrent.locks.ReentrantLock$NonfairSync)
>>>
>>> "Low Memory Detector" daemon prio=10 tid=0x080f9400 nid=0x2ae3
>>> runnable [0x00000000..0x00000000]
>>> java.lang.Thread.State: RUNNABLE
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "CompilerThread1" daemon prio=10 tid=0x080f7400 nid=0x2ae2 waiting
>>> on condition [0x00000000..0x7c984428]
>>> java.lang.Thread.State: RUNNABLE
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "CompilerThread0" daemon prio=10 tid=0x080f4400 nid=0x2ae1 waiting
>>> on condition [0x00000000..0x7ca05468]
>>> java.lang.Thread.State: RUNNABLE
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "Signal Dispatcher" daemon prio=10 tid=0x080f2c00 nid=0x2ae0
>>> runnable [0x00000000..0x7ca56e10]
>>> java.lang.Thread.State: RUNNABLE
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "Finalizer" daemon prio=10 tid=0x080dec00 nid=0x2adf in
>>> Object.wait() [0x7cae9000..0x7caea0b0]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
>>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
>>> - locked <0xb021d838> (a java.lang.ref.ReferenceQueue$Lock)
>>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
>>> at
>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "Reference Handler" daemon prio=10 tid=0x080dd400 nid=0x2ade in
>>> Object.wait() [0x7cb3a000..0x7cb3ae30]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xb021d900> (a java.lang.ref.Reference$Lock)
>>> at java.lang.Object.wait(Object.java:485)
>>> at
>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
>>> - locked <0xb021d900> (a java.lang.ref.Reference$Lock)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "main" prio=10 tid=0x08058800 nid=0x2ada in Object.wait()
>>> [0xb7d71000..0xb7d721f8]
>>> java.lang.Thread.State: WAITING (on object monitor)
>>> at java.lang.Object.wait(Native Method)
>>> - waiting on <0xafa12e60> (a
>>> com.sun.enterprise.v3.server.AppServerStartup$1)
>>> at java.lang.Thread.join(Thread.java:1143)
>>> - locked <0xafa12e60> (a
>>> com.sun.enterprise.v3.server.AppServerStartup$1)
>>> at
>>> com.sun.enterprise.v3.server.AppServerStartup.stop(AppServerStartup.java:348)
>>>
>>> at org.glassfish.api.embedded.Server.stop(Server.java:359)
>>> at
>>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.modifyInitialDomainXml(CreateDomainCommand.java:773)
>>>
>>> at
>>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.createTheDomain(CreateDomainCommand.java:496)
>>>
>>> at
>>> com.sun.enterprise.admin.cli.optional.CreateDomainCommand.runCommand(CreateDomainCommand.java:267)
>>>
>>> at
>>> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:171)
>>>
>>> at
>>> com.sun.enterprise.cli.framework.CLIMain.invokeCommand(CLIMain.java:105)
>>>
>>> at
>>> com.sun.enterprise.admin.cli.AsadminMain.local(AsadminMain.java:190)
>>> at
>>> com.sun.enterprise.admin.cli.AsadminMain.main(AsadminMain.java:84)
>>>
>>> Locked ownable synchronizers:
>>> - None
>>>
>>> "VM Thread" prio=10 tid=0x080d9c00 nid=0x2add runnable
>>>
>>> "GC task thread#0 (ParallelGC)" prio=10 tid=0x08060000 nid=0x2adb
>>> runnable
>>>
>>> "GC task thread#1 (ParallelGC)" prio=10 tid=0x08061400 nid=0x2adc
>>> runnable
>>>
>>> "VM Periodic Task Thread" prio=10 tid=0x080fb000 nid=0x2ae4 waiting
>>> on condition
>>>
>>> JNI global references: 842
>>>
>>>
>>>>
>>>> Thanks,
>>>>
>>>> - Hong
>>>>
>>>>>
>>>>>
>>>>> Hong Zhang wrote:
>>>>>
>>>>>>
>>>>>>> I just built successfully on the Mac using an existing Maven repo.
>>>>>>> Looks like this hang is happening sporadically.
>>>>>>> BTW: are you using a Linux? I recall seeing the same hang in
>>>>>>> create-domain on the Hudson dev job yesterday. Hudson dev job
>>>>>>> is also running on Linux.
>>>>>>>
>>>>>> Yes, I am on ubuntu.
>>>>>>
>>>>>>>
>>>>>>> Hong Zhang wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>>>> in which module is this ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Most times in core/kernel. There was one time the build was
>>>>>>>>>> able to go further, and then when it's building the nucleus
>>>>>>>>>> distribution, it hang with a similar error.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ok let me know if it hangs somewhere else. I need to spawn a
>>>>>>>>> process to test the embedded as running several embedded
>>>>>>>>> instances in the same process can have side effects.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Jerome: I have updated workspace and got your changes. Now it
>>>>>>>> hang in building nucleus distribution:
>>>>>>>>
>>>>>>>> create.distribution:
>>>>>>>> [echo] Creating distribution: nucleus
>>>>>>>> [unzip] Expanding:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/felix-3.0-SNAPSHOT.zip
>>>>>>>> into
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>>
>>>>>>>> [unzip] Expanding:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-grizzly-3.0-SNAPSHOT.zip
>>>>>>>> into
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>>
>>>>>>>> [unzip] Expanding:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-hk2-3.0-SNAPSHOT.zip
>>>>>>>> into
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>>
>>>>>>>> [unzip] Expanding:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfish-nucleus-3.0-SNAPSHOT.zip
>>>>>>>> into
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>>
>>>>>>>> [mkdir] Created dir:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/target
>>>>>>>> [delete] DEPRECATED - Use of the implicit FileSet is
>>>>>>>> deprecated. Use a nested fileset element instead.
>>>>>>>> [delete] Deleting 4 files from
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage
>>>>>>>>
>>>>>>>>
>>>>>>>> fetch-domain-creation-artifacts:
>>>>>>>> [echo] cli-optional jar:
>>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/cli-optional/3.0-SNAPSHOT/cli-optional-3.0-SNAPSHOT.jar
>>>>>>>>
>>>>>>>> [echo] server-mgmt jar:
>>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/server-mgmt/3.0-SNAPSHOT/server-mgmt-3.0-SNAPSHOT.jar
>>>>>>>>
>>>>>>>> [echo] admin-util jar:
>>>>>>>> /home/hzhang/.m2/repository/org/glassfish/admin/admin-util/3.0-SNAPSHOT/admin-util-3.0-SNAPSHOT.jar
>>>>>>>>
>>>>>>>> [echo] security jar:
>>>>>>>> /home/hzhang/.m2/repository/org/glassfish/security/security/3.0-SNAPSHOT/security-3.0-SNAPSHOT.jar
>>>>>>>>
>>>>>>>> [copy] Copying 1 file to
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>>
>>>>>>>> [copy] Copying 1 file to
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>>
>>>>>>>> [copy] Copying 1 file to
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>>
>>>>>>>> [copy] Copying 1 file to
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/modules
>>>>>>>>
>>>>>>>>
>>>>>>>> init:
>>>>>>>>
>>>>>>>> create-domain-on-windows:
>>>>>>>>
>>>>>>>> create-domain-on-unix:
>>>>>>>>
>>>>>>>> create-domain:
>>>>>>>> [echo] Directory to run command from:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin
>>>>>>>>
>>>>>>>> [echo] Command to run:
>>>>>>>> /home/hzhang/files/workspaces/v3/distributions/nucleus/target/stage/glassfishv3/glassfish/bin/asadmin
>>>>>>>>
>>>>>>>> [exec] Using port 4848 for Admin.
>>>>>>>> [exec] Using port 8080 for HTTP Instance.
>>>>>>>> [exec] Using default port 7676 for JMS.
>>>>>>>> [exec] Using default port 3700 for IIOP.
>>>>>>>> [exec] Using default port 8181 for HTTP_SSL.
>>>>>>>> [exec] Using default port 3820 for IIOP_SSL.
>>>>>>>> [exec] Using default port 3920 for IIOP_MUTUALAUTH.
>>>>>>>> [exec] Using default port 8686 for JMX_ADMIN.
>>>>>>>> [exec] Distinguished Name of the self-signed X.509 Server
>>>>>>>> Certificate is:
>>>>>>>> [exec] [CN=localhost,OU=GlassFish,O=Sun
>>>>>>>> Microsystems,L=Santa Clara,ST=California,C=US]
>>>>>>>> [exec] Jul 14, 2009 11:51:55 AM
>>>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>>>> [exec] INFO: GlassFish v3 startup time : Embedded(307ms)
>>>>>>>> startup services(224ms) total(531ms)
>>>>>>>> [exec] Jul 14, 2009 11:51:55 AM AppServerStartup run
>>>>>>>> [exec] INFO: [Thread[GlassFish Kernel Main Thread,5,main]]
>>>>>>>> started
>>>>>>>> [exec] Received Stop
>>>>>>>> [exec] Sending stop event to ms
>>>>>>>>
>>>>>>>>
>>>>>>>> - Hong
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> you updated and did mvn clean install from the root ?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, I updated, and did mvn -U clean install from the root.
>>>>>>>>>>
>>>>>>>>>> - Hong
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Jul 14, 2009, at 8:10 AM, Hong Zhang wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I have updated my workspace this morning and tried to build
>>>>>>>>>>>> the workspace and the build kept hanging. I have
>>>>>>>>>>>> restarted a few times and no difference. There is where
>>>>>>>>>>>> it hangs:
>>>>>>>>>>>>
>>>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>>>> T E S T S
>>>>>>>>>>>> -------------------------------------------------------
>>>>>>>>>>>> Running com.sun.enterprise.v3.admin.AdminAdapterTest
>>>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time
>>>>>>>>>>>> elapsed: 0.019 sec
>>>>>>>>>>>> Running com.sun.enterprise.v3.admin.CreateProfilerTest
>>>>>>>>>>>> Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time
>>>>>>>>>>>> elapsed: 0.001 sec
>>>>>>>>>>>> Running org.glassfish.tests.kernel.embedded.EmbeddedTest
>>>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>>>> com.sun.enterprise.v3.server.AppServerStartup run
>>>>>>>>>>>> INFO: GlassFish v3 startup time : Embedded(219ms)
>>>>>>>>>>>> startup services(170ms) total(389ms)
>>>>>>>>>>>> Jul 14, 2009 10:55:43 AM AppServerStartup run
>>>>>>>>>>>> INFO: [Thread[GlassFish Kernel Main Thread,5,main]] started
>>>>>>>>>>>> Jul 14, 2009 10:55:43 AM
>>>>>>>>>>>> com.sun.enterprise.v3.admin.CommandRunnerImpl getCommand
>>>>>>>>>>>> INFO: Command create-http-listener not found
>>>>>>>>>>>> Received Stop
>>>>>>>>>>>> Sending stop event to ms
>>>>>>>>>>>>
>>>>>>>>>>>> Anyone seeing similar thing? I have svn revision 29132.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> - Hong
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>>------------------------------------------------------------------------
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>