users@glassfish.java.net

Clusterjsp doesn't replicate either [ was : Re: shoal / GMS sniffer? ]

From: Dick Davies <rasputnik_at_hellooperator.net>
Date: Tue, 25 Nov 2008 15:01:04 +0000

Ok, I'm currently testing the cluster by using the clusterjsp EAR and
still having no luck.
Looks like each side can't retrieve session from the other for some reason.
Here's my detailed walkthrough:


So, I've deployed clusterjsp.ear to my 2-node cluster:
      $ asadmin list-instances -p 8048
      devel-roller01 running
      devel-roller02 running
      Command list-instances executed successfully.
      asadmin deploy -p 8048 --contextroot=/clusterjsp \
      --upload=true --enabled=true --precompilejsp=true
--availabilityenabled=true \
      --target roller-cluster \
/data/glassfish/devel/SUNWappserver/samples/quickstart/clusterjsp/clusterjsp.ear


The server instance logs on both backends indicate HA is enabled for the EAR:

[#|2008-11-25T14:52:53.016+0000|INFO|sun-appserver9.1|javax.ee.enterprise.system.tools.synchronization|_ThreadID=34;_ThreadName=RMI
TCP Connection(79)-10.255.224.44;|SYNC062: Synchronization for
clusterjsp is complete. Total time spent 116 milli second(s).|#]

[#|2008-11-25T14:52:53.088+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=34;_ThreadName=RMI
TCP Connection(79)-10.255.224.44;|
IN ReplicatedWebmethodSessionStrategyBuilder NEW|#]

[#|2008-11-25T14:52:53.088+0000|INFO|sun-appserver9.1|javax.enterprise.system.container.web|_ThreadID=34;_ThreadName=RMI
TCP Connection(79)-10.255.224.44;/clusterjsp;replicated;web-method;session;|WEB0130:
Enabling ha-based persistence for web module [/clusterjsp]'s sessions:
persistence-type = [replicated] / persistenceFrequency = [web-method]
/ persistenceScope = [session]|#]

[#|2008-11-25T14:52:53.091+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=34;_ThreadName=RMI
TCP Connection(79)-10.255.224.44;|
JxtaReplicationReceiver>>doPipeInitialization:previously called = true|#]




Set load balancer to only send to mporrlr02 and save 3 name-value
pairs to the session:

       HttpSession Information:
       Served From Server: testblogs
       Server Port Number: 80
       Executed From Server: mporrlr02
       Served From Server instance: devel-roller02
       Executed Server IP Address: 10.255.224.45
       Session ID: 40b0e381597bc46a42826ccc239e
       Session Created: Tue Nov 25 14:26:13 GMT 2008
       Last Accessed: Tue Nov 25 14:26:34 GMT 2008
       Session will go inactive in 1800 seconds

       Data retrieved from the HttpSession:
       name3 = val3
       name2 = val2
       name1 = val1

 mporrlr02 server.log shows

       [#|2008-11-25T14:26:29.085+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=31;_ThreadName=httpSSL
       WorkerThread-38080-12;|Add to session: name1 = val1|#]

       [#|2008-11-25T14:26:34.872+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=32;_ThreadName=httpSSL
       WorkerThread-38080-10;|Add to session: name2 = val2|#]

       [#|2008-11-25T14:26:44.027+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=33;_ThreadName=httpSSL
       WorkerThread-38080-8;|Add to session: name3 = val3|#]


 now flip the backends and click 'reload page' and I see

    HttpSession Information:
    Served From Server: testblogs
    Server Port Number: 80
    Executed From Server: mporrlr01
    Served From Server instance: devel-roller01
    Executed Server IP Address: 10.255.224.44
    Session ID: 413268a92a0b40b5e8b53c077268
    Session Created: Tue Nov 25 14:35:04 GMT 2008
    Last Accessed: Tue Nov 25 14:35:04 GMT 2008
    Session will go inactive in 1800 seconds

    Enter session attribute data:
    Name of Session Attribute:
    Value of Sesion Attribute:



    Data retrieved from the HttpSession:


    and the mporrlr01 server.log shows


    [#|2008-11-25T14:35:00.192+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|transferFromReplicationCache
for id: 40b0e381597bc46a42826ccc239e took: 0 ms|#]

    [#|2008-11-25T14:35:00.193+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|ReplicationStore>>load:localCachedState=null|#]

    [#|2008-11-25T14:35:00.193+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|
    TEST:load called during reshape|#]

    [#|2008-11-25T14:35:04.200+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|findSessionViaBroadcast
for id: 40b0e381597bc46a42826ccc239e took: 4007 ms|#]

    [#|2008-11-25T14:35:04.204+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|
    No parameter entered for this request|#]

and there's nothing at all in mporrlr02s log (the one that has the session).





On Mon, Nov 24, 2008 at 2:37 PM, Dick Davies
<rasputnik_at_hellooperator.net> wrote:
> On Fri, Nov 21, 2008 at 5:47 PM, Shreedhar Ganapathy
> <Shreedhar.Ganapathy_at_sun.com> wrote:
>>
>>
>>
>> I can see multicast traffic ... related to cluster events (instances stopping/starting)
>> but nothing that looks like a session being replicated (or copied
>> over) - even when I
>> tweak the load balancer to send a client to one node, have them log in
>> and then point the LB to the other node.
>
>> Can you restate the steps you followed for the above to help our folks
>> reproduce this behavior?
>> I assume you are following the High Availability guide.
>
> Yes, this is mostly out an out-of-the-box GF cluster, with a Citrix Netscaler
> (HTTP switch) in front of it.
>
>> Just to reaffirm. Your app is availability-enabled and if its a web app it
>> has the <distributable/> tag in the web.xml.
>
> Yes to both.
>
>> And at the moment you are using clusterjsp web app. Right?
>
> No, this is a deploy of the Roller weblogger engine. I'm going to try
> clusterjsp on the development
> cluster tomorrow to see if that has the same issues, I think.
>
>> Do you see any sessions replicated at all or none?
>
> Nothing obvious in the logs, but then no errors either.
> Hence me asking for debugging tools :)
>
>> Session replication component uses GMS only for cluster lifecycle events
>> such as an instance joining, leaving or failing.
>>
>> In-memory replication component also uses Jxta directly for its replication
>> traffic using Jxta's JxtaBiDiPipe which is an bidirectional socket
>> abstraction.
>
> Ah, hadn't realised that; assumed session replication piggybacked on
> GMS traffic.
>
> Does that mean sessions do not go over multicast?
>
>> Does anyone know of any introductory reference material on writing a
>> GMS client?
>
>> http://blogs.sun.com/shreedhar/entry/shoal_clustering_101
>
> Thanks, that's very useful.
>