When I issue "asadmin stop-cluster <cluster-name>" after a week or so of uptime, I can see the following behaviour in each instance's server.log:
[i][#|2009-05-07T01:17:01.718+0800|INFO|sun-appserver9.1|ShoalLogger|_ThreadID=253;_ThreadName=MessageWindowThread;server;|Received Group Shutting down message from member : server|#]
[/i]
This shows the DAS instructing the cluster to shutdown. I wait.
[i]ASync task has not finished. Giving up after 180 seconds
[/i]
This tells me that it's not shutting down smoothly, for some reason that I'm trying to find out right now.
[i][#|2009-05-07T00:38:37.880+0800|INFO|sun-appserver9.1|javax.resourceadapter.mqjmsra.outbound.connection|_ThreadID=223;_ThreadName=Timer-6;|MQJMSRA_MC1101: constructor:Created mcId=30192:xacId=7167796663707117056:Using xacf config={imqDefaultUsername=guest, imqBrokerHostName=localhost, imqSetJMSXUserID=false, imqJMSExpiration=0, imqQueueBrowserRetrieveTimeout=60000, imqSSLProviderClassname=com.sun.net.ssl.internal.ssl.Provider, imqConfiguredClientID=, imqSSLIsHostTrusted=false, imqSetJMSXProducerTXID=false, imqJMSDeliveryMode=PERSISTENT, imqAddressListBehavior=PRIORITY, imqEnableSharedClientID=false, imqAckOnProduce=, imqConnectionURL=
http://localhost/imq/tunnel, imqConnectionFlowLimit=1000, imqSetJMSXAppID=false, imqAddressListIterations=3, imqBrokerServicePort=0, imqSetJMSXRcvTimestamp=false, imqReconnectInterval=5000, imqConnectionType=TCP, imqLoadMaxToServerSession=true, imqSetJMSXConsumerTXID=false, imqAddressList=localhost:37676,mq://CM-app01:37676/,mq://CM-app02:37676/, imqOverrideJMSDeliveryMode=false, imqPingInterval=30, imqDefaultPassword=guest, imqAckOnAcknowledge=, imqJMSPriority=4, imqQueueBrowserMaxMessagesPerRetrieve=1000, imqOverrideJMSPriority=false, imqConsumerFlowLimit=1000, imqBrokerHostPort=7676, imqReconnectEnabled=true, imqConnectionFlowCount=100, imqReconnectAttempts=3, imqConnectionFlowLimitEnabled=false, imqAckTimeout=0, imqBrokerServiceName=, imqOverrideJMSExpiration=false, imqConnectionHandler=com.sun.messaging.jmq.jmsclient.protocol.tcp.TCPStreamHandler, imqDisableSetClientID=false, imqOverrideJMSHeadersToTemporaryDestinations=false, imqConsumerFlowThreshold=50}|#]
[/i]
Intermittently, while waiting I see these messages appearing. This sort of clues me in that there might be a subscriber to a JMS queue that's still latching on to the MQ and is preventing the shutdown (not sure, just my assumption right now).
At the end of the day, I might have to issue the stop-cluster command 3-5 times before it successfully shuts down. I'd be getting the error message "Not alll server instances in cluster <cluster-name> were successfully stopped." a few times until the 4th or 5th time.
Now, I'm trying to trace what exactly is holding up the shutdown. Is it possible to tell from the logs I showed above? I don't see the physical destination of the MQ that's being referenced, so I can't tell (I have quite a number of queues setup). Do I need to do something else, say pause the queues before shutting down?
Would appreciate some pointers to the right direction, thanks!
Wong
[Message sent by forum member 'lilwong' (lilwong)]
http://forums.java.net/jive/thread.jspa?messageID=345259