dev@glassfish.java.net

Re: timing of DAS joining GMS group

From: Shreedhar Ganapathy <Shreedhar.Ganapathy_at_Sun.COM>
Date: Wed, 22 Apr 2009 14:24:13 -0700

cc'd Joe who will be back tomorrow and may have more to add.

Dies Koper wrote:
> Hi,
>
> When using clusters GMS takes care of notifying the participating
> cluster instances of membership changes.
> The DAS also seems to participate in this as spectator, but as the
> processing to join in happens at start up time (see stacktrace below),
> you have to restart the DAS first.
>
You should not have to restart the DAS. When a new cluster is created,
DAS joins that group and being the first in the group becomes the master
node that would own the responsibility of sending out cluster member
lifecycle events. Also, when DAS is shutdown and restarted, it wold
join the GMS group of each cluster defined within the domain.
At what point do you see the stack trace below?
> I'm not sure yet why the DAS needs to be notified of its clusters'
> membership changes,
There is some code in the DAS that would rely on these states to report
health state of clustered instances. asadmin get-health command is one
of the intended ones but I believe currently uses a different code path.
Also these states can be used to show state in monitoring tools like
Performance Monitor and self management action triggers. That was
intention with making DAS the spectator - so it get cluster lifecycle
events but its own failure is not relevant to the cluster.

> but for each cluster the DAS seems to create like 10
> threads so with multiple clusters running the restart of the DAS makes
> it sluggish and consume a lot of resources.
>
Could you share more details on whether this specific set of threads for
each cluster makes DAS sluggish? Some metrics would help.

> Am I looking at a potential bug or is this per design?
>
The notion of a group management service with its associated threads per
cluster is by design. I am not sure if you have seen a stack trace and
if you did, there may be a bug involved. Please send us more details.

> at com.sun.enterprise.jxtamgmt.HealthMonitor.start(HealthMonitor.java:381)
> at com.sun.enterprise.jxtamgmt.ClusterManager.start(ClusterManager.java:339)
> at
> com.sun.enterprise.ee.cms.impl.jxta.GroupCommunicationProviderImpl.join(GroupCommunicationProviderImpl.java:142)
> at com.sun.enterprise.ee.cms.impl.jxta.GMSContext.join(GMSContext.java:124)
> at
> com.sun.enterprise.ee.cms.impl.common.GroupManagementServiceImpl.join(GroupManagementServiceImpl.java:309)
> at
> com.sun.enterprise.ee.cms.lifecycle.GMSLifecycleImpl.initializeGMSForThisInstance(GMSLifecycleImpl.java:217)
> at
> com.sun.enterprise.ee.cms.lifecycle.GMSLifecycleImpl.onInitialization(GMSLifecycleImpl.java:102)
> at
> com.sun.enterprise.server.ApplicationServer.onInitialization(ApplicationServer.java:284)
> at
> com.sun.enterprise.server.ondemand.OnDemandServer.onInitialization(OnDemandServer.java:101)
>
>
>
> Thanks,
> Dies
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>
>