Okay I will look into your current patch (is this Patch A that you just
resent to dev alias?).
Bongjae Chang wrote:
> Hi Shreedhar.
>
> I think that my current patch which I attached recently through
> dev-alias maybe resolved it.
>
> Solution is that though member receives ADD_EVENT, member checks the
> previous view like MASTER_CHANGE_EVENT.
>
> Of course, there can be better solution. But in the current code, I
> think that this solution is the simplest choice without big changes.
>
> But like MASTER_CHANGE_EVENT, there can be additional load. e.g) for
> loop, String.contains(), etc...
>
> But because ADD_EVENT, MASTER_CHANGE_EVENT and JOINED_AND_READY_EVENT
> is occurred rarely, So it seems not bad.
>
> Thanks
> --
> Bongjae Chang
>
>
>
> ----- Original Message -----
> *From:* Shreedhar Ganapathy <mailto:Shreedhar.Ganapathy_at_Sun.COM>
> *To:* users_at_shoal.dev.java.net <mailto:users_at_shoal.dev.java.net>
> *Sent:* Monday, June 30, 2008 3:28 PM
> *Subject:* Re: [Shoal-Users] Still not sure it's working
>
> Hi Bongjae
> You are right about this fix being partial and I saw this earlier
> today while running a simple test.
>
> We do need to think about this a bit more in terms of duplicate
> joins.
> I will look into the provider code tomorrow in MasterNode to see
> how it treats members who are not in the previous view but for
> whom an Add event has not been fired.
> If you get to it sooner, let me know.
>
> Thanks
> Shreedhar
>
> Bongjae Chang wrote:
>> Hi Shreedhar and Mike.
>>
>> I know Shreedhar is waiting for Mike's response about Issue
>> #66(https://shoal.dev.java.net/issues/show_bug.cgi?id=66)
>>
>> But I think maybe this patch is not enough.
>>
>> I expect that Server2 will receive duplicated join notification.
>>
>> Here is my logs.
>>
>> Assume that Server1's server token is
>> 517a0864-1a11-4474-b40a-aff1ce79c12e and Server2's server token
>> is 347f676f-5300-4f40-a409-7ee2d7882c14.
>>
>>
>> [Server1]
>> -------------------------------------------------------------------
>> 2008. 6. 30 오후 1:22:57
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Starting SimpleJoinTest....
>> 2008. 6. 30 오후 1:22:58
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest initializeGMS
>> 정보: Initializing Shoal for member:
>> 517a0864-1a11-4474-b40a-aff1ce79c12e group:TestGroup
>> 2008. 6. 30 오후 1:22:58
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Registering for group event notifications
>> 2008. 6. 30 오후 1:22:58
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Joining Group TestGroup
>> 2008. 6. 30 오후 1:22:59
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>> 정보: GMS View Change Received for group TestGroup : Members in
>> view for (before change analysis) are :
>> 1: MemberId: 517a0864-1a11-4474-b40a-aff1ce79c12e, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEAE124E2EAD144600A358C3E2A889E9F203
>>
>> 2008. 6. 30 오후 1:22:59
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>> 정보: Analyzing new membership snapshot received as part of event
>> : MASTER_CHANGE_EVENT
>> ****JoinNotification received: GroupLeader = false,
>> Signal.getMemberToken() = 517a0864-1a11-4474-b40a-aff1ce79c12e,
>> ServerName = 517a0864-1a11-4474-b40a-aff1ce79c12e, Leader =
>> 517a0864-1a11-4474-b40a-aff1ce79c12e*
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>> 정보: GMS View Change Received for group TestGroup : Members in
>> view for (before change analysis) are :
>> 1: MemberId: 517a0864-1a11-4474-b40a-aff1ce79c12e, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEAE124E2EAD144600A358C3E2A889E9F203
>> 2: MemberId: 347f676f-5300-4f40-a409-7ee2d7882c14, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEC059DA8E0EC34A5696DB9C670EAB932503
>>
>> 2008. 6. 30 오후 1:23:06
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>> 정보: Analyzing new membership snapshot received as part of event
>> : ADD_EVENT
>> ****JoinNotification received: GroupLeader = true,
>> Signal.getMemberToken() = 347f676f-5300-4f40-a409-7ee2d7882c14,
>> ServerName = 517a0864-1a11-4474-b40a-aff1ce79c12e, Leader =
>> 517a0864-1a11-4474-b40a-aff1ce79c12e
>> -------------------------------------------------------------------
>> *
>>
>>
>> [Server2]
>> -------------------------------------------------------------------
>> 2008. 6. 30 오후 1:23:04
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Starting SimpleJoinTest....
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest initializeGMS
>> 정보: Initializing Shoal for member:
>> 347f676f-5300-4f40-a409-7ee2d7882c14 group:TestGroup
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Registering for group event notifications
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.shoal.jointest.SimpleJoinTest runSimpleSample
>> 정보: Joining Group TestGroup
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>> 정보: GMS View Change Received for group TestGroup : Members in
>> view for (before change analysis) are :
>> 1: MemberId: 347f676f-5300-4f40-a409-7ee2d7882c14, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEC059DA8E0EC34A5696DB9C670EAB932503
>>
>> 2008. 6. 30 오후 1:23:05
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>> 정보: Analyzing new membership snapshot received as part of event
>> : MASTER_CHANGE_EVENT
>> ****JoinNotification received: GroupLeader = false,
>> Signal.getMemberToken() = 347f676f-5300-4f40-a409-7ee2d7882c14,
>> ServerName = 347f676f-5300-4f40-a409-7ee2d7882c14, Leader =
>> 347f676f-5300-4f40-a409-7ee2d7882c14
>> *2008. 6. 30 오후 1:23:06
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>> 정보: GMS View Change Received for group TestGroup : Members in
>> view for (before change analysis) are :
>> 1: MemberId: 517a0864-1a11-4474-b40a-aff1ce79c12e, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEAE124E2EAD144600A358C3E2A889E9F203
>> 2: MemberId: 347f676f-5300-4f40-a409-7ee2d7882c14, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEC059DA8E0EC34A5696DB9C670EAB932503
>>
>> 2008. 6. 30 오후 1:23:06
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>> 정보: Analyzing new membership snapshot received as part of event
>> : MASTER_CHANGE_EVENT
>> ****JoinNotification received: GroupLeader = false,
>> Signal.getMemberToken() = 517a0864-1a11-4474-b40a-aff1ce79c12e,
>> ServerName = 347f676f-5300-4f40-a409-7ee2d7882c14, Leader =
>> 517a0864-1a11-4474-b40a-aff1ce79c12e
>> *2008. 6. 30 오후 1:23:06
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>> 정보: GMS View Change Received for group TestGroup : Members in
>> view for (before change analysis) are :
>> 1: MemberId: 517a0864-1a11-4474-b40a-aff1ce79c12e, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEAE124E2EAD144600A358C3E2A889E9F203
>> 2: MemberId: 347f676f-5300-4f40-a409-7ee2d7882c14, MemberType:
>> CORE, Address:
>> urn:jxta:uuid-0836778E36C54F728D5B934A965395CEC059DA8E0EC34A5696DB9C670EAB932503
>>
>> 2008. 6. 30 오후 1:23:06
>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>> 정보: Analyzing new membership snapshot received as part of event
>> : ADD_EVENT
>> ****JoinNotification received: GroupLeader = false,
>> Signal.getMemberToken() = 347f676f-5300-4f40-a409-7ee2d7882c14,
>> ServerName = 347f676f-5300-4f40-a409-7ee2d7882c14, Leader =
>> 517a0864-1a11-4474-b40a-aff1ce79c12e*
>> -------------------------------------------------------------------
>>
>> The problem is the following.
>>
>> Sever1 and Server2 will receive own join notifications by current
>> changes which Shreedhar edited.
>> But after they receive own join notifications from first
>> MASTER_CHANGE_EVENT , Server1 will send Server2 ADD_EVENT because
>> Server1 is the master and is aware of Server2 newly.
>> Then Server2 will receive ADD_EVENT from Server1 which is the
>> master and notify duplicate join notification.
>>
>> So we can know that though any member receives ADD_EVENT from
>> master, the member doesn't have to notify join notification
>> without checking view history's changes. The member should always
>> notify join notification by view histroy's changes like
>> MASTER_CHANGE_EVENT.
>>
>> Or Shreedhar's current patch should be applied to only master.
>>
>> I expect that maybe Mike will obtain similar result. :-)
>>
>> Thanks.
>>
>> --
>> Bongjae Chang
>>
>>
>>
>> ----- Original Message -----
>> *From:* Shreedhar Ganapathy <mailto:Shreedhar.Ganapathy_at_Sun.COM>
>> *To:* users_at_shoal.dev.java.net <mailto:users_at_shoal.dev.java.net>
>> *Sent:* Saturday, June 28, 2008 6:42 AM
>> *Subject:* Re: [Shoal-Users] Still not sure it's working
>>
>> Hi Mike
>> I have checked in a fix to ViewWindow where some wierd logic
>> earlier was excluding the first view change from being notified.
>> Could you try with the latest sources?
>>
>> Thanks
>> Shreedhar
>>
>> Shreedhar Ganapathy wrote:
>>> Thanks for bringing this up. We may have a bug in our
>>> internal test harness which should be reporting these issues
>>> and hence we have not seen it yet. Let me check.
>>> Could you meantime file an issue in the tracker?
>>> I know Bongjae Chang had made some observations on this a
>>> while ago and his patch is awaiting review and integration.
>>> That might have a fix.
>>>
>>> Thanks
>>> Shreedhar
>>>
>>> Mike Wannamaker wrote:
>>>>
>>>> I believe it’s a bug then. In both accounts. My SERVER-1
>>>> only gets a SERVER-1 message and SERVER-2 only gets a
>>>> SERVER-1 message.
>>>>
>>>> So when I say it only gets a SERVER-1 message I mean that
>>>> the method
>>>>
>>>> public void processNotification(Signal p_Signal)
>>>>
>>>> is only being called with that message.
>>>>
>>>> So if I start just SERVER-1, I see the GMS View Changed
>>>> message, with just the SERVER-1 in it, but my
>>>> processNotification(…) is not called. Not until I start
>>>> SERVER-2 does it get called.
>>>>
>>>> On SERVER-2, I see the original GMS View Changed with just
>>>> SERVER-2, and then GMS View Changed with SERVER-1,SERVER-2,
>>>> but only get one processNotification(…) call.
>>>>
>>>> I will investigate further next week, but if you could have
>>>> a look that would be great. Is no one else seeing this?
>>>>
>>>> This is my processNotification() method
>>>>
>>>> public void processNotification(Signal p_Signal)
>>>>
>>>> {
>>>>
>>>> try
>>>>
>>>> {
>>>>
>>>> p_Signal.acquire();
>>>>
>>>> SignalLogger log = new
>>>> SignalLogger(p_Signal);
>>>>
>>>> log.logIt();
>>>>
>>>> if(p_Signal instanceof
>>>> MessageSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> MessageSignal msgSig = (MessageSignal)p_Signal;
>>>>
>>>> String
>>>> sMember = msgSig.getMemberToken();
>>>>
>>>> Object o =
>>>> ObjectUtil.toObject(msgSig.getMessage());
>>>>
>>>> if(o
>>>> instanceof SMessage)
>>>>
>>>> {
>>>>
>>>>
>>>> SMessage smsg = (SMessage)o;
>>>>
>>>>
>>>> InetAddress sender = m_hmMembers.get(sMember).address;
>>>>
>>>>
>>>> smsg.setSender(sender);
>>>>
>>>>
>>>> SMessageLogger.log.systemInfo(getClass(), "FireMessage: " +
>>>> smsg);
>>>>
>>>>
>>>> fireMessageReceived(smsg);
>>>>
>>>>
>>>> //fireMessageReceived(smsg);
>>>>
>>>> }
>>>>
>>>> else
>>>>
>>>> {
>>>>
>>>>
>>>> SMessageLogger.log.systemInfo(getClass(), "Message is NOT
>>>> SMessage??");
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof JoinNotificationSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> JoinNotificationSignal joinSig =
>>>> (JoinNotificationSignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof JoinedAndReadyNotificationSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> JoinedAndReadyNotificationSignal joinSig =
>>>> (JoinedAndReadyNotificationSignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof FailureSuspectedSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> FailureSuspectedSignal suspectSig =
>>>> (FailureSuspectedSignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof FailureRecoverySignal)
>>>>
>>>> {
>>>>
>>>>
>>>> FailureRecoverySignal failureSig =
>>>> (FailureRecoverySignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof FailureNotificationSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> FailureNotificationSignal failureSig =
>>>> (FailureNotificationSignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else if(p_Signal
>>>> instanceof PlannedShutdownSignal)
>>>>
>>>> {
>>>>
>>>>
>>>> PlannedShutdownSignal shutdownSig =
>>>> (PlannedShutdownSignal)p_Signal;
>>>>
>>>>
>>>> processClusterNotification();
>>>>
>>>> }
>>>>
>>>> else
>>>>
>>>> {
>>>>
>>>>
>>>> SMessageLogger.log.debug(getClass(), "Received Notification
>>>> of type : " + p_Signal.getClass().getName() + " Server: "
>>>> + p_Signal.getMemberToken());
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> catch(SignalAcquireException e)
>>>>
>>>> {
>>>>
>>>>
>>>> SMessageLogger.log.fatal(getClass(), "Exception occured
>>>> while acquiring signal", e);
>>>>
>>>> }
>>>>
>>>> finally
>>>>
>>>> {
>>>>
>>>> try
>>>>
>>>> {
>>>>
>>>>
>>>> p_Signal.release();
>>>>
>>>> }
>>>>
>>>>
>>>> catch(SignalReleaseException e)
>>>>
>>>> {
>>>>
>>>>
>>>> SMessageLogger.log.warn(getClass(), "Exception occured
>>>> while releasing signal" , e);
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> }
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> *From:* Shreedhar.Ganapathy_at_Sun.COM
>>>> [mailto:Shreedhar.Ganapathy_at_Sun.COM]
>>>> *Sent:* June 27, 2008 11:21 AM
>>>> *To:* users_at_shoal.dev.java.net
>>>> *Subject:* Re: [Shoal-Users] Still not sure it's working
>>>>
>>>> Hi Mike
>>>> The expected behavior is that as each server starts, its
>>>> registered GMS client components will be notified of the
>>>> server's own joining the group and any subsequent joins of
>>>> other members.
>>>> So in essence, server-1 GMS clients should see a
>>>> JoinNotificationSignal for server-1, and another for server-2
>>>> and in server-2, GMS clients should see a
>>>> JoinNotificationSignal for server-2 and another for server-1.
>>>> The order here does not matter but correctness is important
>>>> and if not its a bug to be fixed.
>>>>
>>>> In the log below, Server-1 seems to be getting its own
>>>> JoinNotificationSignal which is correct. Does it ever get
>>>> the JoinNotificationSignal for server-2?
>>>> On server-2, I am seeing correct behavior.
>>>>
>>>> (Ignore the log statements that show the view contents, as
>>>> that is an event coming from the provider implementation -
>>>> GMS notification signals are the ones that GMS clients
>>>> should look in for correctness).
>>>>
>>>> Let me know.
>>>> Thanks
>>>> Shreedhar
>>>>
>>>> Mike Wannamaker wrote:
>>>>
>>>> Hi Guys,
>>>>
>>>> I’m still not sure it’s working as it’s supposed to? But
>>>> maybe it is?
>>>>
>>>> Start SERVER-1
>>>>
>>>> Start SERVER-2
>>>>
>>>> On SERVER-1 I get a JoinMessage but it is from
>>>> SERVER-1?
>>>>
>>>> On SERVER-2 I get a Join Message from SERVER-1,
>>>> which is what I would expect?
>>>>
>>>> Is this correct? This depends on when the two servers are
>>>> started. If I wait for a period between startups I get
>>>> SERVER-2 startup message on SERVER-1 and SERVER-1 startup
>>>> message on SERVER-2. But if I start them both at the same
>>>> time I get the above behaviour?
>>>>
>>>> Starting both at virtually the same time I get …
>>>>
>>>> SEVER-1 Output:
>>>>
>>>> 27-Jun-2008 12:40:29 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>>>>
>>>> INFO: GMS View Change Received for group RCS_CLUSTER :
>>>> Members in view for (before change analysis) are :
>>>>
>>>> 1: MemberId: SERVER-1, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FDB946A28335F0413BBF73B77CCC8BFEC603
>>>>
>>>> 27-Jun-2008 12:40:29 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>>>>
>>>> INFO: Analyzing new membership snapshot received as part of
>>>> event : MASTER_CHANGE_EVENT
>>>>
>>>> 27-Jun-2008 12:40:38 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>>>>
>>>> INFO: GMS View Change Received for group RCS_CLUSTER :
>>>> Members in view for (before change analysis) are :
>>>>
>>>> 1: MemberId: SERVER-2, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FD0D4B867250FF460C9B539A161779845B03
>>>>
>>>> 2: MemberId: SERVER-1, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FDB946A28335F0413BBF73B77CCC8BFEC603
>>>>
>>>> 27-Jun-2008 12:40:38 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>>>>
>>>> INFO: Analyzing new membership snapshot received as part of
>>>> event : ADD_EVENT
>>>>
>>>> 27-Jun-2008 12:40:38 AM DEBUG [pool-1-thread-1]
>>>> com.opentext.ecm.services.smessage.impl.shoal.SignalLogger
>>>> - - SERVER-1 >> JoinNotificationSignalImpl @ 27/06/08 12:40
>>>> AM - [RCS_CLUSTER]:
>>>> (Hashtable:[(String:server.name)<-->(String:SERVER-1),
>>>> (String:local.host)<-->(Inet4Address:mwana0061/10.6.2.89)])
>>>>
>>>> Server-2 Output
>>>>
>>>> 27-Jun-2008 12:40:30 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>>>>
>>>> INFO: GMS View Change Received for group RCS_CLUSTER :
>>>> Members in view for (before change analysis) are :
>>>>
>>>> 1: MemberId: SERVER-2, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FD0D4B867250FF460C9B539A161779845B03
>>>>
>>>> 27-Jun-2008 12:40:30 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>>>>
>>>> INFO: Analyzing new membership snapshot received as part of
>>>> event : MASTER_CHANGE_EVENT
>>>>
>>>> 27-Jun-2008 12:40:38 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow getMemberTokens
>>>>
>>>> INFO: GMS View Change Received for group RCS_CLUSTER :
>>>> Members in view for (before change analysis) are :
>>>>
>>>> 1: MemberId: SERVER-2, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FD0D4B867250FF460C9B539A161779845B03
>>>>
>>>> 2: MemberId: SERVER-1, MemberType: CORE, Address:
>>>> urn:jxta:uuid-2F39FF376B6A43E3905DAFC81B7D02FDB946A28335F0413BBF73B77CCC8BFEC603
>>>>
>>>> 27-Jun-2008 12:40:38 AM
>>>> com.sun.enterprise.ee.cms.impl.jxta.ViewWindow newViewObserved
>>>>
>>>> INFO: Analyzing new membership snapshot received as part of
>>>> event : ADD_EVENT
>>>>
>>>> 27-Jun-2008 12:40:44 AM DEBUG [pool-1-thread-1]
>>>> com.opentext.ecm.services.smessage.impl.shoal.SignalLogger
>>>> - - SERVER-1 >> JoinNotificationSignalImpl @ 27/06/08 12:40
>>>> AM - [RCS_CLUSTER]:
>>>> (Hashtable:[(String:server.name)<-->(String:SERVER-1),
>>>> (String:local.host)<-->(Inet4Address:mwana0061/10.6.2.89)])
>>>>