users@glassfish.java.net

Re: Clusterjsp doesn't replicate either [ was : Re: shoal / GMS sniffer? ]

From: Shreedhar Ganapathy <Shreedhar.Ganapathy_at_Sun.COM>
Date: Tue, 25 Nov 2008 07:17:01 -0800

Hi Dick
It appears from the log below that the switch may not have sent out the
jsession cookie and jsession version id headers when the request was
failed over to the second instance.
Here's what you can do.

    * Send in one request through your browser pointing to the switch url.
    * Check your browser cookie cache to see if a jsessionid cookie was
      generated
    * From the clusterjsp app response to the first request, you will
      know the instance that served the request.
    * Stop that instance.
    * Send in the request again with additional session data
    * Check this time in the browser cookie cache if the jsessionid
      exists and its jversionid string has an incremented number.
    * This time the request should have failed over to the second
      instance and you should see session values from first request
      reproduced in the response.

If you do not see the sessions created with the first request, then the
switch should be looked into for passing in the appropriate headers for
session retrieval.

Hope this is helpful.
--Shreedhar

Dick Davies wrote:
> Ok, I'm currently testing the cluster by using the clusterjsp EAR and
> still having no luck.
> Looks like each side can't retrieve session from the other for some reason.
> Here's my detailed walkthrough:
>
>
> So, I've deployed clusterjsp.ear to my 2-node cluster:
> $ asadmin list-instances -p 8048
> devel-roller01 running
> devel-roller02 running
> Command list-instances executed successfully.
> asadmin deploy -p 8048 --contextroot=/clusterjsp \
> --upload=true --enabled=true --precompilejsp=true
> --availabilityenabled=true \
> --target roller-cluster \
> /data/glassfish/devel/SUNWappserver/samples/quickstart/clusterjsp/clusterjsp.ear
>
>
> The server instance logs on both backends indicate HA is enabled for the EAR:
>
> [#|2008-11-25T14:52:53.016+0000|INFO|sun-appserver9.1|javax.ee.enterprise.system.tools.synchronization|_ThreadID=34;_ThreadName=RMI
> TCP Connection(79)-10.255.224.44;|SYNC062: Synchronization for
> clusterjsp is complete. Total time spent 116 milli second(s).|#]
>
> [#|2008-11-25T14:52:53.088+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=34;_ThreadName=RMI
> TCP Connection(79)-10.255.224.44;|
> IN ReplicatedWebmethodSessionStrategyBuilder NEW|#]
>
> [#|2008-11-25T14:52:53.088+0000|INFO|sun-appserver9.1|javax.enterprise.system.container.web|_ThreadID=34;_ThreadName=RMI
> TCP Connection(79)-10.255.224.44;/clusterjsp;replicated;web-method;session;|WEB0130:
> Enabling ha-based persistence for web module [/clusterjsp]'s sessions:
> persistence-type = [replicated] / persistenceFrequency = [web-method]
> / persistenceScope = [session]|#]
>
> [#|2008-11-25T14:52:53.091+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=34;_ThreadName=RMI
> TCP Connection(79)-10.255.224.44;|
> JxtaReplicationReceiver>>doPipeInitialization:previously called = true|#]
>
>
>
>
> Set load balancer to only send to mporrlr02 and save 3 name-value
> pairs to the session:
>
> HttpSession Information:
> Served From Server: testblogs
> Server Port Number: 80
> Executed From Server: mporrlr02
> Served From Server instance: devel-roller02
> Executed Server IP Address: 10.255.224.45
> Session ID: 40b0e381597bc46a42826ccc239e
> Session Created: Tue Nov 25 14:26:13 GMT 2008
> Last Accessed: Tue Nov 25 14:26:34 GMT 2008
> Session will go inactive in 1800 seconds
>
> Data retrieved from the HttpSession:
> name3 = val3
> name2 = val2
> name1 = val1
>
> mporrlr02 server.log shows
>
> [#|2008-11-25T14:26:29.085+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=31;_ThreadName=httpSSL
> WorkerThread-38080-12;|Add to session: name1 = val1|#]
>
> [#|2008-11-25T14:26:34.872+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=32;_ThreadName=httpSSL
> WorkerThread-38080-10;|Add to session: name2 = val2|#]
>
> [#|2008-11-25T14:26:44.027+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=33;_ThreadName=httpSSL
> WorkerThread-38080-8;|Add to session: name3 = val3|#]
>
>
> now flip the backends and click 'reload page' and I see
>
> HttpSession Information:
> Served From Server: testblogs
> Server Port Number: 80
> Executed From Server: mporrlr01
> Served From Server instance: devel-roller01
> Executed Server IP Address: 10.255.224.44
> Session ID: 413268a92a0b40b5e8b53c077268
> Session Created: Tue Nov 25 14:35:04 GMT 2008
> Last Accessed: Tue Nov 25 14:35:04 GMT 2008
> Session will go inactive in 1800 seconds
>
> Enter session attribute data:
> Name of Session Attribute:
> Value of Sesion Attribute:
>
>
>
> Data retrieved from the HttpSession:
>
>
> and the mporrlr01 server.log shows
>
>
> [#|2008-11-25T14:35:00.192+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|transferFromReplicationCache
> for id: 40b0e381597bc46a42826ccc239e took: 0 ms|#]
>
> [#|2008-11-25T14:35:00.193+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|ReplicationStore>>load:localCachedState=null|#]
>
> [#|2008-11-25T14:35:00.193+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|
> TEST:load called during reshape|#]
>
> [#|2008-11-25T14:35:04.200+0000|INFO|sun-appserver9.1|com.sun.enterprise.ee.web.sessmgmt|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|findSessionViaBroadcast
> for id: 40b0e381597bc46a42826ccc239e took: 4007 ms|#]
>
> [#|2008-11-25T14:35:04.204+0000|INFO|sun-appserver9.1|javax.enterprise.system.stream.out|_ThreadID=21;_ThreadName=httpSSLWorkerThread-38080-18;|
> No parameter entered for this request|#]
>
> and there's nothing at all in mporrlr02s log (the one that has the session).
>
>
>
>
>
> On Mon, Nov 24, 2008 at 2:37 PM, Dick Davies
> <rasputnik_at_hellooperator.net> wrote:
>
>> On Fri, Nov 21, 2008 at 5:47 PM, Shreedhar Ganapathy
>> <Shreedhar.Ganapathy_at_sun.com> wrote:
>>
>>>
>>> I can see multicast traffic ... related to cluster events (instances stopping/starting)
>>> but nothing that looks like a session being replicated (or copied
>>> over) - even when I
>>> tweak the load balancer to send a client to one node, have them log in
>>> and then point the LB to the other node.
>>>
>>> Can you restate the steps you followed for the above to help our folks
>>> reproduce this behavior?
>>> I assume you are following the High Availability guide.
>>>
>> Yes, this is mostly out an out-of-the-box GF cluster, with a Citrix Netscaler
>> (HTTP switch) in front of it.
>>
>>
>>> Just to reaffirm. Your app is availability-enabled and if its a web app it
>>> has the <distributable/> tag in the web.xml.
>>>
>> Yes to both.
>>
>>
>>> And at the moment you are using clusterjsp web app. Right?
>>>
>> No, this is a deploy of the Roller weblogger engine. I'm going to try
>> clusterjsp on the development
>> cluster tomorrow to see if that has the same issues, I think.
>>
>>
>>> Do you see any sessions replicated at all or none?
>>>
>> Nothing obvious in the logs, but then no errors either.
>> Hence me asking for debugging tools :)
>>
>>
>>> Session replication component uses GMS only for cluster lifecycle events
>>> such as an instance joining, leaving or failing.
>>>
>>> In-memory replication component also uses Jxta directly for its replication
>>> traffic using Jxta's JxtaBiDiPipe which is an bidirectional socket
>>> abstraction.
>>>
>> Ah, hadn't realised that; assumed session replication piggybacked on
>> GMS traffic.
>>
>> Does that mean sessions do not go over multicast?
>>
>>
>>> Does anyone know of any introductory reference material on writing a
>>> GMS client?
>>>
>>> http://blogs.sun.com/shreedhar/entry/shoal_clustering_101
>>>
>> Thanks, that's very useful.
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>
>