Hi Wolfram,
Wolfram Rittmeyer wrote:
> Nandini Ektare wrote:
>> Hi Wolfram,
>>
>> This looks like the issue
>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=3289
>>
>> It has been fixed on GF v2. You can instead take that patch on GF.
>
> Hi Nandini,
>
> I've done a fresh install of glassfish-v2-b50g and of
> glassfish-installer-v2ur1-b09d and the issue is reproducible easily.
> Just change the address-attribute of the jmx-connector element to a
> specific address and the following stack trace gets printed in
> server.log (see log below). The same is true for a freshly build of
> SJSAS91_FCS_BRANCH. Applying the path I sent to this list yesterday
> fixes the issue (though granted there might be more elegant ways to
> deal with the issue).
>
> Could it be that the fix you mentioned is not ported to the v2.1 branch?
Hmm.. depends on when this branch forked off. I see I checked in for
3289 between 2007-07-16 and 2007-08-01
If you can verify any one of the following diffs, it will be clear.
http://fisheye5.cenqua.com/browse/glassfish/jmx-remote/rjmx-impl/src/java/com/sun/enterprise/admin/jmx/remote/server/rmi/RmiStubRegistryHandler.java?r1=1.6&r2=1.7
http://fisheye5.cenqua.com/browse/glassfish/jmx-remote/rjmx-impl/src/java/com/sun/enterprise/admin/jmx/remote/server/rmi/JmxConnectorServerDriver.java?r1=1.8&r2=1.9
http://fisheye5.cenqua.com/browse/glassfish/appserv-core/src/java/com/sun/enterprise/admin/server/core/JmxConnectorLifecycle.java?r1=1.15&r2=1.16
http://fisheye5.cenqua.com/browse/glassfish/admin-ee/nodeagent/src/java/com/sun/enterprise/ee/nodeagent/NodeAgent.java?r1=1.27&r2=1.28
Also, I went back and saw closing comments for 3289 :
<snip>
This bug (DAS fails to start with a specific IP for system-jmx-connector) can
be reproduced consistently only on Developer profile (PE) with
ASQuickStart=true (the default value for Developer profile)
With ASQuickStart=false it is not reproducible.
The flag is not considered in Cluster profile, so no matter what value you set
for the flag for Cluster profile, it is not reproducible (which is why I
couldn't reproduce it anytime.)
</snip>
Can you please vary the value of this flag and report your observation?
Lastly, to give you more details of what your fix achieves and what
3289's fix does:
- Fix for 3289 uproots the issues right at the point where
JMXConnectorServer finally binds to a port. The connector server is
nothing but a RMI server object and socket creation for it is controlled
by GlassFish using custom server socket factories. All the custom server
socket factories have the 0.0.0.0 vs specified port address logic. So it
would be surprising why you should observe otherwise
- Your patch fixes the JMXServiceUrl used by JDK's JMX code in creating
the JMXConnectorServer. I remember at one point bug reporter for 3289
confirmed that fixing URL did not fix the issue which is why I am not so
sure about including the patch. At that time, since I could not change
what was being done within JDK, I had finally resorted to modifying the
last point in control flow just before socket creation in
GlassFish-provided server socket factory (JDK delegates control to the
custom socket factory if one exists).
Please let me know if you see the 3289 diffs in the branch you are
working on and your observation for the ASQuickStart flag.
Regards,
Nandini
> Regards,
>
> Wolfram Rittmeyer
> P.S.: Quicklook results:
> [java] [java] ************************
> [java] [java] PASSED= 69
> [java] [java] ------------ =========
> [java] [java] FAILED= 0
> [java] [java] ------------ =========
> [java] [java] DID NOT RUN= 0
> [java] [java] ------------ =========
> [java] [java] Total Expected=69
> [java] [java] ************************
>
>
>
> server.log:
>
> [#|2008-04-20T00:47:03.968+0200|INFO|sun-appserver9.1|javax.enterprise.system.tools.admin|_ThreadID=10;_ThreadName=main;|ADM1079:
> Initialization of AMX MBeans started|#]
>
> [#|2008-04-20T00:47:04.264+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;com.sun.enterprise.admin.server.core.JmxConnectorLifecycle_at_247d4a;com.sun.appserv.server.ServerLifecycleException:
> Cannot bind to URL
> [rmi://localhost:8686/management/rmi-jmx-connector]:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost;
> nested exception is:
> java.net.ConnectException: Connection
> refused];_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Service
> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle_at_247d4a
> cannot be started! : com.sun.appserv.server.ServerLifecycleException:
> Cannot bind to URL
> [rmi://localhost:8686/management/rmi-jmx-connector]:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost;
> nested exception is:
> java.net.ConnectException: Connection refused]|#]
>
> [#|2008-04-20T00:47:04.269+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5071:
> An error occured during initialization
> com.sun.appserv.server.ServerLifecycleException: Cannot bind to URL
> [rmi://localhost:8686/management/rmi-jmx-connector]:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost;
> nested exception is:
> java.net.ConnectException: Connection refused]
> at
> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:153)
>
> at
> com.sun.enterprise.server.ApplicationServer.onStartup(ApplicationServer.java:442)
>
> at
> com.sun.enterprise.server.ondemand.OnDemandServer.onStartup(OnDemandServer.java:120)
>
> at com.sun.enterprise.server.PEMain.run(PEMain.java:411)
> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
> Caused by: java.io.IOException: Cannot bind to URL
> [rmi://localhost:8686/management/rmi-jmx-connector]:
> javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost;
> nested exception is:
> java.net.ConnectException: Connection refused]
> at
> javax.management.remote.rmi.RMIConnectorServer.newIOException(RMIConnectorServer.java:814)
>
> at
> javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:431)
>
> at
> com.sun.enterprise.admin.jmx.remote.server.rmi.JmxConnectorServerDriver.startConnectorServer(JmxConnectorServerDriver.java:218)
>
> at
> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onStartup(JmxConnectorLifecycle.java:134)
>
> ... 9 more
> Caused by: javax.naming.ServiceUnavailableException [Root exception is
> java.rmi.ConnectException: Connection refused to host: localhost;
> nested exception is:
> java.net.ConnectException: Connection refused]
> at
> com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:122)
> at
> com.sun.jndi.toolkit.url.GenericURLContext.bind(GenericURLContext.java:208)
>
> at javax.naming.InitialContext.bind(InitialContext.java:359)
> at
> javax.management.remote.rmi.RMIConnectorServer.bind(RMIConnectorServer.java:635)
>
> at
> javax.management.remote.rmi.RMIConnectorServer.start(RMIConnectorServer.java:427)
>
> ... 11 more
> Caused by: java.rmi.ConnectException: Connection refused to host:
> localhost; nested exception is:
> java.net.ConnectException: Connection refused
> at
> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:574)
> at
> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:185)
> at
> sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:171)
> at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:306)
> at sun.rmi.registry.RegistryImpl_Stub.bind(Unknown Source)
> at
> com.sun.jndi.rmi.registry.RegistryContext.bind(RegistryContext.java:116)
> ... 15 more
> Caused by: java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
> at
> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at
> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl$2.run(ASPlainSocketImpl.java:419)
>
> at java.security.AccessController.doPrivileged(Native Method)
> at
> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl._invoke(ASPlainSocketImpl.java:416)
>
> at
> com.sun.enterprise.server.ss.provider.ASPlainSocketImpl.connect(ASPlainSocketImpl.java:288)
>
> at
> com.sun.enterprise.server.ss.provider.ASClientSocketImpl.connect(ASClientSocketImpl.java:267)
>
> at java.net.Socket.connect(Socket.java:516)
> at java.net.Socket.connect(Socket.java:466)
> at java.net.Socket.<init>(Socket.java:366)
> at java.net.Socket.<init>(Socket.java:179)
> at
> sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
>
> at
> sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
>
> at
> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:569)
> ... 20 more
> |#]
>
> [#|2008-04-20T00:47:04.287+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Server
> Startup failed. Exiting...|#]
>
> [#|2008-04-20T00:47:04.288+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|Server
> shutdown in progress...|#]
>
> [#|2008-04-20T00:47:04.288+0200|INFO|sun-appserver9.1|javax.enterprise.system.container.web|_ThreadID=10;_ThreadName=main;|WEB0303:
> Stopping Sun-Java-System/Application-Server.|#]
>
> [#|2008-04-20T00:47:04.289+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
> Exception :
> com.sun.appserv.server.ServerLifecycleException: WEB0106: An error
> occurred while stopping the web container
> at
> com.sun.enterprise.web.PEWebContainer.stopInstance(PEWebContainer.java:822)
>
> at
> com.sun.enterprise.web.PEWebContainerLifecycle.onShutdown(PEWebContainerLifecycle.java:95)
>
> at
> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>
> at
> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>
> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
> Caused by: LifecycleException: WEB0103: This web container has not
> yet been started
> at
> com.sun.enterprise.web.WebContainer.stop(WebContainer.java:913)
> at
> com.sun.enterprise.web.PEWebContainer.stopInstance(PEWebContainer.java:814)
>
> ... 10 more
> |#]
>
> [#|2008-04-20T00:47:04.290+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE5051:
> Shutting down all J2EE applications ...|#]
>
> [#|2008-04-20T00:47:04.290+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE
> 5060 : Stopping all J2EE Connector 1.5 Compliant resource adapters
> ...|#]
>
> [#|2008-04-20T00:47:04.292+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE
> 5061 : Resource adapters Stop() Complete|#]
>
> [#|2008-04-20T00:47:04.293+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;|CORE5052:
> Application shutdown complete.|#]
>
> [#|2008-04-20T00:47:04.293+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
> Exception :
> java.lang.NullPointerException
> at
> com.sun.enterprise.util.InvocationManagerImpl.preInvoke(InvocationManagerImpl.java:136)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.preInvoke(ServerLifecycleModule.java:178)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.postEvent(ServerLifecycleModule.java:219)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.onShutdown(ServerLifecycleModule.java:273)
>
> at
> com.sun.appserv.server.LifecycleModuleService.onShutdown(LifecycleModuleService.java:487)
>
> at
> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>
> at
> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>
> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
> |#]
>
> [#|2008-04-20T00:47:04.303+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
> Exception :
> com.sun.appserv.server.ServerLifecycleException
> at
> com.sun.enterprise.admin.server.core.JmxConnectorLifecycle.onShutdown(JmxConnectorLifecycle.java:171)
>
> at
> com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
>
> at
> com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
>
> at com.sun.enterprise.server.PEMain.run(PEMain.java:429)
> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
> |#]
>
> [#|2008-04-20T00:47:04.872+0200|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|CORE5061:
> Exception :
> java.lang.NullPointerException
> at
> com.sun.enterprise.util.InvocationManagerImpl.preInvoke(InvocationManagerImpl.java:136)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.preInvoke(ServerLifecycleModule.java:178)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.postEvent(ServerLifecycleModule.java:219)
>
> at
> com.sun.appserv.server.ServerLifecycleModule.onTermination(ServerLifecycleModule.java:277)
>
> at
> com.sun.appserv.server.LifecycleModuleService.onTermination(LifecycleModuleService.java:496)
>
> at
> com.sun.enterprise.server.ApplicationServer.onTermination(ApplicationServer.java:587)
>
> at com.sun.enterprise.server.PEMain.run(PEMain.java:434)
> at com.sun.enterprise.server.PEMain.main(PEMain.java:338)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>
> at java.lang.reflect.Method.invoke(Method.java:585)
> at com.sun.enterprise.server.PELaunch.main(PELaunch.java:412)
> |#]
>
> [#|2008-04-20T00:47:04.873+0200|SEVERE|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=10;_ThreadName=main;_RequestID=0324a2d5-9ad6-416d-b6aa-a122b9380c38;|Server
> stopped due to Server startup failure.|#]
>
> [#|2008-04-20T00:47:04.874+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=12;_ThreadName=Thread-1;|sending
> notification to server...server|#]
>
> [#|2008-04-20T00:47:04.876+0200|INFO|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=12;_ThreadName=Thread-1;|Server
> shutdown complete.|#]
>
>
>>
>> Regards,
>> Nandini
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>