users@shoal.java.net

Re: [Shoal-Users] is shoal fitting my need and how to set it up correctly (black listing IP addresses)

From: Shreedhar Ganapathy <Shreedhar.Ganapathy_at_Sun.COM>
Date: Sat, 28 Jun 2008 09:41:27 -0700

Hi Christian
Good questions. Hope the following gives you a good perspective on
whether on not Shoal's Distributed State Cache can help you.

Christian Brennsteiner wrote:
> hi all,
>
> i am just trying to find out if shoal would be a good fitting solution
> to my problem.
>
> architecture:
> sun java appserver (glassfish 2.2)
I hope this is v2 or v2 Update 2
> 10 appservers each has 20 EAR files deployed
> each ear file should know a hasmap of blacklisted IP adresses (~100
> addresses)
> 5 appservers on server 1 seperated from 5 appservers on server 2 via
> WAN (low latency)
I suppose, this is a cluster of 10 appserver instances divvied up on two
machines and all 20 enterprise apps are deployed on this cluster. Or is
it such that each of the 10 instances will have 2 enterprise apps each
and the instances are not clustered in the GF v2 clustering sense ?

Will the WAN allow multicast traffic ? With Shoal version 1.0ur, there
is no support for cross subnet traffic. You will need to enable at the
router(s) multicast traffic on a designated address and port.
>
> goal:
> every ear file should be able to maintain a central black list where a
> list of bad IP addresses is kept.
Typically such solutions can be had through Hardware/Software
Loadbalancers ( not the one shipping with GlassFish) which take care of
intrusion detection and exclusion lists ( for banned IP addresses) in
addition to providing SSL accelerators and sophisticated ways to ensure
high throughput traffic with fast failover. You might want to look into
that option as well.
> is shoal capeable of managing such distributed state? if yes which
> point to start first whats the best entrypoint for me?
Shoal has a distributed state cache (DSC) object per cluster/group which
is good for lightweight communications. It is available through the
GroupManagementService.getGroupHandle().getDistributedCache().
> is there a very easy way to keep the hashmap on each ear file with the
> same information ?
The DSC in Shoal 1.0ur is per group and not available per app yet. We
are looking into that for Shoal 2.0. All apps in each VM have access to
the DSC.

You can of course have an app level hashmap of your own, and use the
Shoal group messaging APIs to send and receive messages on your hashmap
updates. This will give you partitioning at the app level if that is
what you want to achieve.

Look in GroupManagementService.getGroupHandle().sendMessage() apis and
the MessageAction/MessageActionFactory/MessageSignal constructs.
> if yes what happens when wan goes down and ear 1 on server 1 adds an
> entry will ear 2 on server 1 see this?
Since the DSC in 1.0ur is global, there is no app level partitioning of
data in the DSC although each entry can have a "component name" field
that can be used to identify the app. If ear1 and ear2 are on the same
vm they should definitely see each others' entries. If each ear is on a
separate appserver instance, then networking will be required. As long
as subnet level networking is available apps on other appserver
instances should be able to see entries on server1 and similarly on
server 2
Also, if you set your loopback address to a non 127.0.0.x address (ex.
192.168.x.x), members on the same machine should be able to see DSC
entries without networking available.

However, with the WAN down, clients to your app may not have access to
your app.
> what is about the hashmaps on server 2 ? will the stay the same when
> network WAN goes down and server 2 does not see server 1 anymore?
> (split brain?)
Without networking, cache copies on each instance cannot be synchronized.
> what happens if wan goes up again and server 1 discovers server 2 again?
The DSC instances should start seeing each other again and it cache
syncs start happening again and if the data is static, you should not
have any stale data issues.
DSC in its current form does not have any semantics of which piece of
data is more preferable to overwrite another piece of data coming from
different instances. It is a simple cache and last write to the same
composite key (serverName, componentName, Key combination) would
overwrite previous writes.
>
> is there a central place where such things are documented in shoal?
We are beginning to wrt these aspects. We don't yet have a formal
documentation resource to help us and are looking for community help in
this area. Based on incoming user feedback, we are currently building
documents that can cover all use cases.
>
> regards chris