Hi Shreedhar Ganapathy
Thanks for your reply.
Now, I am simply reviewing Shoal code. Up to now, I don't have special use cases for only DSC.
But, I hope that I use DSC for Web's distributed Session, Stateful Session Bean, JNDI cache, etc... in the future. :-)
Return to the main, I think Shoal used to utilize DSC for protective failure fencing operations.
I think following Shoal's source code is a important thing about protective failure fencing operations.
---
<In GroupHandleImpl.java>
public void raiseFence(final String componentName, final String failedMemberToken) throws GMSException {
if (!isFenced(componentName, failedMemberToken)) {
...
// update informations
dsc.addToCache(componentName, getGMSContext().getServerIdentityToken(), failedMemberToken, setStateAndTime());
...
}
}
public boolean isFenced(final String componentName, final String memberToken) {
...
while (members.size() > 1 && !dsc.isFirstSyncDone()) {
logger.log(Level.FINE, "Waiting for DSC first Sync");
try {
Thread.sleep(SYNC_WAIT);
count++;
if (count > 4) {
forceDSCSync((DistributedStateCacheImpl) dsc);
}
}
catch (InterruptedException e) {
logger.log(Level.WARNING, e.getLocalizedMessage());
}
}
...
// get informations
entries = dsc.getFromCache(memberToken);
...
}
---
I think dsc.isFirstSyncDone() is true when forceDSCSync() is called like above code and new group is joined.( is it right? )
If dsc.isFirstSyncDone() is false, the group leader(coordinator) will deliver DSC to all members after all. Then there is no problem.
But if all members have true state from dsc.isFirstSyncDone() and raiseFence() method is called concurrently for each member,
I have a doubt that some problem can be occurred about protective failure fencing operations.
ex) When one more members can execute raiseFence() or isFenced() concurrently, DSC will be updated or returned by another informations. Then one more members will be able to execute recovery logic concurrently.
So I had a question that DSC guarantees data consistency and same view for same group concurrently.
I want to see the error of my ways. please point out mistakes.
thanks!
--
Bongjae Chang
----- Original Message -----
From: Shreedhar Ganapathy
To: dev_at_shoal.dev.java.net
Sent: Sunday, February 10, 2008 1:45 AM
Subject: Re: [Shoal-Dev] Shoal DSC(distributed state cache)'s consistency question
Hi Bongjae Chang
Thanks for posting your questions. Please see inline for my responses.
Thanks
Shreedhar
Bongjae Chang wrote:
Hi all. I'm a beginner in Shoal Project.
Recently, I have reviewed Shoal's source code because I am interested in Generic Clustering Framework.
Very glad to know of this. Please let us know of your use cases and we can surely help.
I know DSC uses point-to-point communication(JXTA) with all instances for reliablity.
Up until now DSC implementation was using UDP broadcasts which are inherently unreliable. We are moving to make any message intended to be sent synchronously to use TCP based unicast for reliability and ordering for all messaging in Shoal with the current service provider. That is in the works and not yet complete. That should be happening next week.
following Shoal's source code is a message sending part in GroupCommunicationProviderimpl.java
---
public void sendMessage(...) {
...
List<String> currentMembers = getGMSContext().getGroupHandle().getAllCurrentMembers();
for (String currentMember : currentMembers) {
...
clusterManager.send(id, message);
}
...
}
---
The above code snippet is for applications that want to send messages to other members' application layer. For corresponding DSC send message snippet, look here:
https://shoal.dev.java.net/source/browse/shoal/gms/src/java/com/sun/enterprise/ee/cms/impl/jxta/DistributedStateCacheImpl.java?rev=1.17&view=markup
The above, I have a question in DSC(distributed state cache, default implementation)
Does Shoal DSC guarantee data consistency for same group?
The answer is not a clear yes or no and is "it depends" as total consistency requires total ordering. The service provider impl does not provide total ordering as we have not seen a requirement for it. For DSC, at this point, the lightweight nature of its intended usage does not require total ordering and consistency provided by the transport layer has been sufficient so far in our tests.
As Shoal's adoption increases, we are beginning to see requirements that would help us prioritize our efforts to provide such aspects.
for example,
Current members consist of A, B and C.
When A send a message(key="a", value="1") to the group,
B send another message(but same key, key="a", value="2") to the group at the same time.
At this time, can the group memers(A, B, C) view same value about key="a" in DSC? (all "1" or all "2" )
There is definitely a small timing factor to be considered in any distributed system of this nature. Assuming the network is not a limiting factor, the data should be consistently available in the instances of DSC in each member. More below:
I think it doesn't matter about value(whether "1" or not), but it's important whether group members have same view concurrently or not.
The members are expected to have identical membership view consistently.As a result, messages are delivered to all members in a current view.
And does DSC guarantee Atomicity?(All sent or not)
The current DSC implementation is a simple shared concurrent hashmap. A write from one member gets disseminated to all members so that reads are local. A message sent is not transactional in that there is no notion of a 2-phase commit equivalent.
We have not gone to the level of guaranteeing each of the ACID properties as yet.
For instance, in your example above, with the current implementation, when members A and B write data to the same key, the last member writing data at that moment would overwrite the previous value in the group. That said, I would even venture to say that there is no code in the DSC implementation that would ensure that a single write blocks all other writes for the same key.
Please let us know more details of what your requirements are and we will surely look into providing extensions for such a use case. We are currently in discussions on doing a full fledged ShoalCache and would like to open this up to the community for contributions both for discussions, input and for code.
If DSC guarantees consistency and atomicity, please explain the mechanism briefly :-)
Or please let me know the web link about this information.
I need your helps. Thanks!
Would be glad to help. Keep sending us your questions, enhancement requests and any contributions you may have.
--
Bongjae Chang