dev@glassfish.java.net

Re: SGES v2.1 patch 03: JMX Connector and Client Hostname ... information needed for debugging purpose

From: Kedar Mhaswade <Kedar.Mhaswade_at_Sun.COM>
Date: Tue, 08 Sep 2009 17:37:41 -0700

Vincent,

First off, great code sleuthing on your part. Thanks to you.

I am someone who barely knows how this was supposed to work, but haven't
actually written this code. That fact combined with the "age" of this
code makes my comments vulnerable from a correctness standpoint, but
still ...

Short answer: This code change might not work in case of the "general"
case of non-multihomed machines and no virtual private networks.
This is because the value of "address" attribute of jmx-connector for
server instances in general case is always "0.0.0.0" and that means
the admin server (DAS, the one that tells the GUI about what the
status of instance is) does not know what the current physical address
of the instance is.

In that general case, it was decided that the physical address be
gotten from the Node Agent that controls the server instance. The
property named "client-hostname" was designed for this purpose and
when node-agent shakes the hands with DAS, this property gets
the value.

In the case of multi-homed machines, all of this is not required
because the "address" attribute of a server is going to have the
correct IP address the server end of the JMX connector (inside
the server instance) is going to bind to. In this case, like you
have found out, the "client-hostname" property is not required.

This all boils down to the multi-homed correctness of GlassFish,
something I sadly accept, we should have handled rather better.

What you could try is to actually change the value of client-hostname
on Node-agent that controls your servers to be the same as "address"
of server instance and see if the status determination behaves.

Hope that helps.

Kedar

PS - you should file a bug so this can be looked into.



vincent.konieczek_at_mpsa.com wrote:
> Hello,
>
>
> I am looking for information about the JMX connectors used in GlassFish and
> the way they are used.
>
> In fact, I am interested in a very specific case.
> I have one domain (IP1), one node agent (IP2) and two instances (IP3 and
> IP4) to illustrate a bit.
> IP1, IP2, IP3 and IP4 are all different IP addresses.
>
> With such a setup, the domain, the node agent and the two instances can
> start and every applications deployed on the instances as well.
> Unfortunately when we take a look at the instances status through the admin
> console, we can observe that most of them are inconsistent (indicated as
> Stopped whereas they are running).
> An instance whose status is consistent have an IP address that can be
> resolved to the IP address of the node agent (IP2).
>
> Through several tests, we have observed that the port configured in the
> Admin Service JMX Connector "system" was correctly opened for listening on
> the right IP address.
> On the other side, we have found several "Connection refused" messages in
> the logs that lead us to consider the class
> com.sun.enterprise.config.serverbeans.ServerHelper and the method
> getJMXConnectorInfo(ConfigContext configContext, String instanceName).
> In that method, we can find this :
> //Take the client connection properties from the node agent
> referenced by the
> //given server instance. In other words, the host, user,
> password used to connect to the
> //server instance should be identical to the host, user,
> password used to connect to the
> //server's node agent.
> final JmxConnector agentConnector = NodeAgentHelper.
> getNodeAgentSystemConnector(
> configContext, server.getNodeAgentRef());
> hostProp = agentConnector.getElementPropertyByName(
> HOST_PROPERTY_NAME);
>
> When a JMX connector is needed to connect to the admin service of an
> instance, the hostname used to connect is the hostname of the node agent,
> instead of the hostname of the instance.
>
> We have made a very small patch to modify that behaviour and force the use
> the hostname of the instance in that specific case.
> After putting this patch in the appropriate classpath-prefix and after a
> fresh start of GlassFish, we have observed that the inconsistencies of the
> statuses reported through the admin console have disappeared.
> The admin console or GlassFish seem to work correctly. We have not
> identified problems due to this change (everything that was working still
> works).
> (We have also added log messages to identify the execution paths involved
> by this patch and possible impacts on those paths, and found none).
>
> I would be interested to know a bit more about the reasons behind the
> comment included.
> So if any one has this information or any information related, I would
> kindly appreciate if it could share it with us.
>
> Why does such a restriction (to connect to the instance through the node
> agent host name) exist ?
> And what could be the impacts (or kind of impacts) due to such a change in
> that behaviour ?
>
> Is this a behaviour modification that could be integrated as-is in
> GlassFish or would it need to consider some configuration parameters
> addition to be introduced ?
> If this could lead to a patch for further integration in GlassFish, what
> would be the requirements ?
>
>
>
> About the very small patch ...
> We have replaced this line
> hostProp = agentConnector.getElementPropertyByName(
> HOST_PROPERTY_NAME);
> by the following one in the small extract above.
> hostProp = connector.getElementPropertyByName(
> HOST_PROPERTY_NAME);
>
>
> Thanks in advance for any help on this.
>
>
>
>
> Best Regards
>
>
> Vincent KONIECZEK
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>