Wolfram Rittmeyer wrote:
> Nandini Ektare wrote:
>> Hi Wolfram,
>>
>> Wolfram Rittmeyer wrote:
>>> Nandini Ektare wrote:
>>>> Hi Wolfram,
>>>>
>>>> This looks like the issue
>>>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=3289
>>>>
>>>> It has been fixed on GF v2. You can instead take that patch on GF.
>>> Hi Nandini,
>>>
>>> I've done a fresh install of glassfish-v2-b50g and of
>>> glassfish-installer-v2ur1-b09d and the issue is reproducible easily.
>>> Just change the address-attribute of the jmx-connector element to a
>>> specific address and the following stack trace gets printed in
>>> server.log (see log below). The same is true for a freshly build of
>>> SJSAS91_FCS_BRANCH. Applying the path I sent to this list yesterday
>>> fixes the issue (though granted there might be more elegant ways to
>>> deal with the issue).
>>>
>>> Could it be that the fix you mentioned is not ported to the v2.1
>>> branch?
>> Hmm.. depends on when this branch forked off. I see I checked in for
>> 3289 between 2007-07-16 and 2007-08-01
>> If you can verify any one of the following diffs, it will be clear.
>> http://fisheye5.cenqua.com/browse/glassfish/jmx-remote/rjmx-impl/src/java/com/sun/enterprise/admin/jmx/remote/server/rmi/RmiStubRegistryHandler.java?r1=1.6&r2=1.7
>>
>> http://fisheye5.cenqua.com/browse/glassfish/jmx-remote/rjmx-impl/src/java/com/sun/enterprise/admin/jmx/remote/server/rmi/JmxConnectorServerDriver.java?r1=1.8&r2=1.9
>>
>> http://fisheye5.cenqua.com/browse/glassfish/appserv-core/src/java/com/sun/enterprise/admin/server/core/JmxConnectorLifecycle.java?r1=1.15&r2=1.16
>>
>> http://fisheye5.cenqua.com/browse/glassfish/admin-ee/nodeagent/src/java/com/sun/enterprise/ee/nodeagent/NodeAgent.java?r1=1.27&r2=1.28
>>
>
Hi Wolfram,
> Hi Nandini,
>
> these are all part of main trunk and these changes are part of the
> sources I used to verify the problem described on the glassfish forum.
> Thus the problem is not fixed in the main trunk!
>
> Also setting ASQuickStartup to false doesn't change anything.
Yes, it shouldn't. Otherwise it would mean that bug has not been fixed :-)
Looks like this is a different issue.
> Now by digging a bit deeper into the sources I've understood the
> following:
> In the prepare-method of JmxConnectorServerDriver a ServerSocket on
> the JMX-port is opened and set to "LISTEN". This doesn't cause an
> exception to be thrown.
Yes, 3289's analysis concluded similarly. What looked like an issue for
JMX-port turned out to be an issue for the dynamic port. (the message is
a general message when internal JMX RMI fails to start a connector
server. So that message can be misleading).
> Actually - being no JMX-server developer - I hadn't expected a
> ServerSocket to be opened that early. I would have expected this to
> happen a little bit later.
I am not sure I understand your question. I will be glad to help you
understand the flow of code if you are interested.
> Nevertheless the exception gets thrown in the start()-method of the
> JMXConnectorServer. Why is that? After all the ServerSocket has
> already been opened successfully.
> Two more observations:
> The URL passed to the creation of the JMXConnectorServer points to
> localhost - even though I have specified an explicit IP address in
> domain.xml, which is not surprising given the way the URLs are created.
>
> The env-object passed to the creation of the JMXConnectorServer
> contains a reference to the RMIServerSocketFactory-object created in
> prepare. Yet this object's createServerSocket()-method has already
> been called in prepare and after that gets not called again.
That was the registry's SSF not the connector server's SSF
Since I can't reproduce, let's try to see what is different on your
setup due to which connector server start fails. (I will contact you
offline to get those details)
Regards,
Nandini
> BTW: I have tested this on Linux with a Glassfish developer profile.
> Given the above observations it looks to me like the URL is relevant
> after all. But I guess this issue needs quite a lot of testing
> (different operating systems, different profiles and so on). Thus my
> rather optimistic patch is probably only a first step ;-)
>
>
> Regards,
>
> Wolfram Rittmeyer
>
>
>>
>> Also, I went back and saw closing comments for 3289 :
>>
>> <snip>
>>
>> This bug (DAS fails to start with a specific IP for
>> system-jmx-connector) can be reproduced consistently only on
>> Developer profile (PE) with ASQuickStart=true (the default value for
>> Developer profile)
>> With ASQuickStart=false it is not reproducible.
>>
>> The flag is not considered in Cluster profile, so no matter what
>> value you set for the flag for Cluster profile, it is not
>> reproducible (which is why I couldn't reproduce it anytime.)
>> </snip>
>>
>> Can you please vary the value of this flag and report your observation?
>>
>> Lastly, to give you more details of what your fix achieves and what
>> 3289's fix does:
>> - Fix for 3289 uproots the issues right at the point where
>> JMXConnectorServer finally binds to a port. The connector server is
>> nothing but a RMI server object and socket creation for it is
>> controlled by GlassFish using custom server socket factories. All the
>> custom server socket factories have the 0.0.0.0 vs specified port
>> address logic. So it would be surprising why you should observe
>> otherwise
>> - Your patch fixes the JMXServiceUrl used by JDK's JMX code in
>> creating the JMXConnectorServer. I remember at one point bug reporter
>> for 3289 confirmed that fixing URL did not fix the issue which is why
>> I am not so sure about including the patch. At that time, since I
>> could not change what was being done within JDK, I had finally
>> resorted to modifying the last point in control flow just before
>> socket creation in GlassFish-provided server socket factory (JDK
>> delegates control to the custom socket factory if one exists).
>>
>> Please let me know if you see the 3289 diffs in the branch you are
>> working on and your observation for the ASQuickStart flag.
>>
>> Regards,
>> Nandini
>>> Regards,
>>>
>>> Wolfram Rittmeyer
>>> P.S.: Quicklook results:
>>> [java] [java] ************************
>>> [java] [java] PASSED= 69
>>> [java] [java] ------------ =========
>>> [java] [java] FAILED= 0
>>> [java] [java] ------------ =========
>>> [java] [java] DID NOT RUN= 0
>>> [java] [java] ------------ =========
>>> [java] [java] Total Expected=69
>>> [java] [java] ************************
>>>
>>>
>>>
>>> server.log:
>>>
>>> [#|2008-04-20T00:47:03.968+0200|INFO|sun-appserver9.1|javax.enterprise.system.tools.admin|_ThreadID=10;_ThreadName=main;|ADM1079:
>>> Initialization of AMX MBeans started|#]
>>>
>>> [#|2008-04-20T00:47:04.264+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;com.sun.enterprise.admin.server.core.JmxConnectorLifecycle_at_247d4a;com.sun.appserv.server.ServerLifecycleException:
>>> Cannot bind to URL
>>> [rmi://localhost:8686/management/rmi-jmx-connector]:
>>> javax.naming.ServiceUnavailableException [Root exception is
>>> java.rmi.ConnectException: Connection refused to host: localhost;
>>> nested exception is:
>>> java.net.ConnectException: Connection
>>> refused];_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Service
>>> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle_at_247d4a
>>> cannot be started! :
>>> com.sun.appserv.server.ServerLifecycleException: Cannot bind to URL
>>> [rmi://localhost:8686/management/rmi-jmx-connector]:
>>> javax.naming.ServiceUnavailableException [Root exception is
>>> java.rmi.ConnectException: Connection refused to host: localhost;
>>> nested exception is:
>>> java.net.ConnectException: Connection refused]|#]
>>>
>>> [#|2008-04-20T00:47:04.269+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5071:
>>> An error occured during initialization
>>> com.sun.appserv.server.ServerLifecycleException: Cannot bind to URL
>>> [rmi://localhost:8686/management/rmi-jmx-connector]:
>>> javax.naming.ServiceUnavailableException [Root exception is
>>> java.rmi.ConnectException: Connection refused to host: localhost;
>>> nested exception is:
>>> java.net.ConnectException: Connection refused]
>>> at
>>> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:153)
>>>
>>> at
>>> com.sun.enterprise.server.ApplicationServer.onStartup(ApplicationServer.java:442)
>>>
>>> at
>>> com.sun.enterprise.server.ondemand.OnDemandServer.onStartup(OnDemandServer.java:120)
>>>
>>> at com.sun.enterprise.server.PEMain.run(PEMain.java:411)
>>> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
>>> Caused by: java.io.IOException: Cannot bind to URL
>>> [rmi://localhost:8686/management/rmi-jmx-connector]:
>>> javax.naming.ServiceUnavailableException [Root exception is
>>> java.rmi.ConnectException: Connection refused to host: localhost;
>>> nested exception is:
>>> java.net.ConnectException: Connection refused]
>>> at
>>> javax.management.remote.rmi.RMIConnectorServer.newIOException(RMIConnectorServer.java:814)
>>>
>>> at
>>> javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:431)
>>>
>>> at
>>> com.sun.enterprise.admin.jmx.remote.server.rmi.JmxConnectorServerDriver.startConnectorServer(JmxConnectorServerDriver.java:218)
>>>
>>> at
>>> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:134)
>>>
>>> ... 9 more
>>> Caused by: javax.naming.ServiceUnavailableException [Root exception
>>> is java.rmi.ConnectException: Connection refused to host: localhost;
>>> nested exception is:
>>> java.net.ConnectException: Connection refused]
>>> at
>>> com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:122)
>>>
>>> at
>>> com.sun.jndi.toolkit.url.GenericURLContext.bind(GenericURLContext.java:208)
>>>
>>> at javax.naming.InitialContext.bind(InitialContext.java:359)
>>> at
>>> javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:635)
>>>
>>> at
>>> javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:427)
>>>
>>> ... 11 more
>>> Caused by: java.rmi.ConnectException: Connection refused to host:
>>> localhost; nested exception is:
>>> java.net.ConnectException: Connection refused
>>> at
>>> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:574)
>>> at
>>> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:185)
>>> at
>>> sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
>>> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:306)
>>> at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source)
>>> at
>>> com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:116)
>>>
>>> ... 15 more
>>> Caused by: java.net.ConnectException: Connection refused
>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>>> at
>>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
>>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
>>> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at
>>> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl$2.run(ASPlainSocketImpl.java:419)
>>>
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at
>>> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl._invoke(ASPlainSocketImpl.java:416)
>>>
>>> at
>>> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl.connect(ASPlainSocketImpl.java:288)
>>>
>>> at
>>> com.sun.enterprise.server.ss.provider.ASClientSocketImpl.connect(ASClientSocketImpl.java:267)
>>>
>>> at java.net.Socket.connect(Socket.java:516)
>>> at java.net.Socket.connect(Socket.java:466)
>>> at java.net.Socket.<init>(Socket.java:366)
>>> at java.net.Socket.<init>(Socket.java:179)
>>> at
>>> sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
>>>
>>> at
>>> sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
>>>
>>> at
>>> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:569)
>>> ... 20 more
>>> |#]
>>>
>>> [#|2008-04-20T00:47:04.287+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Server
>>> Startup failed. Exiting...|#]
>>>
>>> [#|2008-04-20T00:47:04.288+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|Server
>>> shutdown in progress...|#]
>>>
>>> [#|2008-04-20T00:47:04.288+0200|INFO|sun-appserver9.1|javax.enterprise.system.container.web|_ThreadID=10;_ThreadName=main;|WEB0303:
>>> Stopping Sun-Java-System/Application-Server.|#]
>>>
>>> [#|2008-04-20T00:47:04.289+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
>>> Exception :
>>> com.sun.appserv.server.ServerLifecycleException: WEB0106: An error
>>> occurred while stopping the web container
>>> at
>>> com.sun.enterprise.web.PEWebContainer.stopInstance(PEWebContainer.java:822)
>>>
>>> at
>>> com.sun.enterprise.web.PEWebContainerLifecycle.onShutdown(PEWebContainerLifecycle.java:95)
>>>
>>> at
>>> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>>>
>>> at
>>> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>>>
>>> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
>>> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
>>> Caused by: LifecycleException: WEB0103: This web container has not
>>> yet been started
>>> at
>>> com.sun.enterprise.web.WebContainer.stop(WebContainer.java:913)
>>> at
>>> com.sun.enterprise.web.PEWebContainer.stopInstance(PEWebContainer.java:814)
>>>
>>> ... 10 more
>>> |#]
>>>
>>> [#|2008-04-20T00:47:04.290+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE5051:
>>> Shutting down all J2EE applications ...|#]
>>>
>>> [#|2008-04-20T00:47:04.290+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE
>>> 5060 : Stopping all J2EE Connector 1.5 Compliant resource adapters
>>> ...|#]
>>>
>>> [#|2008-04-20T00:47:04.292+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE
>>> 5061 : Resource adapters Stop() Complete|#]
>>>
>>> [#|2008-04-20T00:47:04.293+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE5052:
>>> Application shutdown complete.|#]
>>>
>>> [#|2008-04-20T00:47:04.293+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
>>> Exception :
>>> java.lang.NullPointerException
>>> at
>>> com.sun.enterprise.util.InvocationManagerImpl.preInvoke(InvocationManagerImpl.java:136)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.preInvoke(ServerLifecycleModule.java:178)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.postEvent(ServerLifecycleModule.java:219)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.onShutdown(ServerLifecycleModule.java:273)
>>>
>>> at
>>> com.sun.appserv.server.LifecycleModuleService.onShutdown(LifecycleModuleService.java:487)
>>>
>>> at
>>> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>>>
>>> at
>>> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>>>
>>> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
>>> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
>>> |#]
>>>
>>> [#|2008-04-20T00:47:04.303+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
>>> Exception :
>>> com.sun.appserv.server.ServerLifecycleException
>>> at
>>> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onShutdown(JmxConnectorLifecycle.java:171)
>>>
>>> at
>>> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>>>
>>> at
>>> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>>>
>>> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
>>> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
>>> |#]
>>>
>>> [#|2008-04-20T00:47:04.872+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
>>> Exception :
>>> java.lang.NullPointerException
>>> at
>>> com.sun.enterprise.util.InvocationManagerImpl.preInvoke(InvocationManagerImpl.java:136)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.preInvoke(ServerLifecycleModule.java:178)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.postEvent(ServerLifecycleModule.java:219)
>>>
>>> at
>>> com.sun.appserv.server.ServerLifecycleModule.onTermination(ServerLifecycleModule.java:277)
>>>
>>> at
>>> com.sun.appserv.server.LifecycleModuleService.onTermination(LifecycleModuleService.java:496)
>>>
>>> at
>>> com.sun.enterprise.server.ApplicationServer.onTermination(ApplicationServer.java:587)
>>>
>>> at com.sun.enterprise.server.PEMain.run(PEMain.java:434)
>>> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>
>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
>>> |#]
>>>
>>> [#|2008-04-20T00:47:04.873+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Server
>>> stopped due to Server startup failure.|#]
>>>
>>> [#|2008-04-20T00:47:04.874+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=12;_ThreadName=Thread-1;|sending
>>> notification to server...server|#]
>>>
>>> [#|2008-04-20T00:47:04.876+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=12;_ThreadName=Thread-1;|Server
>>> shutdown complete.|#]
>>>
>>>
>>>>
>>>> Regards,
>>>> Nandini
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>