users@glassfish.java.net

testing clustering within xen enterprise

From: <glassfish_at_javadesktop.org>
Date: Fri, 01 Jun 2007 03:00:29 PDT

We're trying to set up a test platform of glassfish/AS9.1 Beta2 in a cluster running within multiple RHEL guests in Xen. Basic config is this:

Xen Enterprise Host 1
RHEL4-GUEST1 - DAS
RHEL4-GUEST2 - Node1

Xen Enterprise Host 2
RHEL-GUEST1 - Node3
RHEL-GUEST2 - Node4

...but it doesn't work. The nodes all fail to perform an initial sync with the DAS and refuse to start. If the DAS is running on Win2k3 they work fine, if everything is on seperate physical boxes/RHEL they work fine, and if I create a node in the same VM as the DAS, it runs ok too. DNS is configured correctly on all guests, and they can all ping each other.

Interestingly the node rendezvous happens just fine - the node shows up in the DAS, but as I say it won't start. DAS and nodes are all currently running as root, though we've tested as non-root with the same results.

My question is has anyone got such a platform to behave in a XenEnt3.2/RHEL4/as9.1 environment? It's fairly important as we're looking to use a similar structure in production.

Thanks in advance + here's an example of a node error log:

buildCommand
INFO:
/usr/java/jdk1.5.0_12/bin/java
-XX:+UnlockDiagnosticVMOptions
-XX:LogFile=/opt/SUNWappserver/nodeagents/node.app3/agent/logs/jvm.log
-XX:+LogVMOutput
-Dcom.sun.aas.configRoot=/opt/SUNWappserver/config
-Dcom.sun.aas.defaultLogFile=/opt/SUNWappserver/nodeagents/node.app3/agent/logs/server.log
-Dcom.sun.aas.instanceName=node.app3
-Dcom.sun.aas.instanceRoot=/opt/SUNWappserver/nodeagents/node.app3/agent
-Dcom.sun.aas.isNodeAgent=true
-Dcom.sun.aas.promptForIdentity=true
-Dcom.sun.appserv.admin.pluggable.features=com.sun.enterprise.ee.admin.pluggable.EEClientPluggableFeatureImpl
-Dcom.sun.appserv.nss.db=/opt/SUNWappserver/nodeagents/node.app3/agent/config
-Dcom.sun.appserv.pluggable.features=com.sun.enterprise.ee.server.pluggable.EEPluggableFeatureImpl
-Djava.endorsed.dirs=/opt/SUNWappserver/lib/endorsed
-Djava.library.path=/opt/SUNWappserver/lib:/opt/SUNWappserver/lib:/opt/SUNWappserver/lib
-Djava.security.auth.login.config=/opt/SUNWappserver/nodeagents/node.app3/agent/config/login.conf
-Djava.util.logging.manager=com.sun.enterprise.server.logging.ServerLogManager
-Djmx.invoke.getters=true
-cp
/opt/SUNWappserver/lib/javaee.jar:/opt/SUNWappserver/lib/appserv-admin.jar:/opt/SUNWappserver/lib/appserv-ext.jar:/opt/SUNWappserver/lib/appserv-se.jar:/opt/SUNWappserver/lib/appserv-rt.jar:/opt/SUNWappserver/lib/appserv-ee.jar
com.sun.enterprise.ee.nodeagent.NodeAgentMain
start
 startInstancesOverride=NOTSET
syncInstances=false
Jun 1, 2007 9:36:05 AM com.sun.enterprise.admin.servermgmt.launch.ASLauncher process
INFO: goodbye
[#|2007-06-01T09:36:05.666+0100|INFO|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|IdentityManager Data: User:admin|#]

[#|2007-06-01T09:36:05.759+0100|INFO|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|NAGT0004:Starting Node Agent...|#]

[#|2007-06-01T09:36:05.775+0100|CONFIG|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|DAS url = service:jmx:rmi:///jndi/rmi://vm-gbx-app2.cimex.com:50613/management/rmi-jmx-connector|#]

[#|2007-06-01T09:36:05.869+0100|INFO|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|NAGT0025:The node agent could not configure logging levels. Default logging level will be used.|#]

[#|2007-06-01T09:36:05.870+0100|INFO|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|NAGT0038:Executing Synchronization for node-agent With DAS|#]

[#|2007-06-01T09:36:14.968+0100|INFO|sun-appserver9.1|javax.ee.enterprise.system.tools.synchronization|_ThreadID=10;_ThreadName=main;|SYNC001: Unable to communicate with Domain Administration Server. Skipping synchronization.|#]

[#|2007-06-01T09:36:14.970+0100|SEVERE|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|NAGT0035:The NodeAgent failed to complete the intial synchronization with the DAS. Please make sure the DAS is running and is accessible from the NodeAgents server|#]

[#|2007-06-01T09:36:16.972+0100|WARNING|sun-appserver9.1|javax.ee.enterprise.system.nodeagent|_ThreadID=10;_ThreadName=main;|NAGT0013:Stopping Node Agent...|#]
[Message sent by forum member 'petermurray' (petermurray)]

http://forums.java.net/jive/thread.jspa?messageID=219932