[#|2012-05-31T08:54:29.041-0700|INFO|44.0|com.sun.enterprise.server.logging.GFFileHandler|_ThreadID=1;_ThreadName=Thread-2;|Running GlassFish Version: Oracle GlassFish Server 4.0 (build 39)|#] [#|2012-05-31T08:54:29.041-0700|INFO|44.0|com.sun.enterprise.server.logging.GFFileHandler|_ThreadID=1;_ThreadName=Thread-2;|GlassFis is using Log Formatter: com.sun.enterprise.server.logging.UniformLogFormatter|#] [#|2012-05-31T08:54:29.629-0700|INFO|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|GMS1095: member: clustered_instance_3 is joining group: sqe-cluster ...|#] [#|2012-05-31T08:54:29.647-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|instantiated following NetworkManager implementation:com.sun.enterprise.mgmt.transport.grizzly.grizzly2.GrizzlyNetworkManager2|#] [#|2012-05-31T08:54:29.650-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|GMS MAX_MESSAGE_LENGTH=4,196,352|#] [#|2012-05-31T08:54:29.650-0700|INFO|44.0|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=10;_ThreadName=Thread-2;|Grizzly 2.0 NetworkManager|#] [#|2012-05-31T08:54:29.654-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;| GrizzlyNetworkManager2 Configuration BIND_INTERFACE_ADDRESS:null NetworkInterfaceName:null TCPSTARTPORT..TCPENDPORT:9090..9200 MULTICAST_ADDRESS:MULTICAST_PORT:228.9.71.122:14955 MULTICAST_PACKET_SIZE:65536 MULTICAST_TIME_TO_LIVE: default FAILURE_DETECT_TCP_RETRANSMIT_TIMEOUT(ms):10000 MAX_PARALLEL:15 START_TIMEOUT(ms):15000 WRITE_TIMEOUT(ms):10000 MAX_WRITE_SELECTOR_POOL_SIZE:30 |#] [#|2012-05-31T08:54:29.729-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|Grizzly controller listening on /0:0:0:0:0:0:0:0:9103. Transport started in 13 ms|#] [#|2012-05-31T08:54:29.734-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|Set via default minimum: MulticastSocket.getTimeToLive()=4|#] [#|2012-05-31T08:54:29.735-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|MulticastSocket configuration: local socket address: 0.0.0.0/0.0.0.0:14955 network interface: name:null multicast address:/228.9.71.122 timeToLive=4|#] [#|2012-05-31T08:54:29.747-0700|CONFIG|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|HealthMonitor: heartBeatTimeout(ms)=2000 maxMissedBeats=3 failureDetectionTCPTimeout(ms)=10000 failureDetectionTCPPort=9000|#] [#|2012-05-31T08:54:29.748-0700|INFO|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|starting cluster sqe-cluster for member:clustered_instance_3|#] [#|2012-05-31T08:54:29.748-0700|INFO|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|GMS1061: MasterNode message listener is registered for member: clustered_instance_3 group: sqe-cluster|#] [#|2012-05-31T08:54:29.754-0700|INFO|44.0|javax.org.glassfish.gms.org.glassfish.gms|_ThreadID=10;_ThreadName=Thread-2;|GMSAD1005: Member clustered_instance_3 joined group sqe-cluster|#] [#|2012-05-31T08:54:29.755-0700|INFO|44.0|javax.org.glassfish.gms.org.glassfish.gms|_ThreadID=10;_ThreadName=Thread-2;|GMSAD1004: Started GMS for instance clustered_instance_3 in group sqe-cluster|#] [#|2012-05-31T08:54:29.769-0700|CONFIG|44.0|ShoalLogger|_ThreadID=12;_ThreadName=Thread-2;|MySystemAdvertisement(summary): GMSMember name: clustered_instance_3 group: sqe-cluster memberType: CORE startTime: May 31, 2012 8:54:29 AM PDT ID:10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 TCP uri(s):[tcp://[fe80:0:0:0:223:8bff:fe64:d810%2]:4000, tcp://10.133.184.142:4000]|#] [#|2012-05-31T08:54:29.793-0700|CONFIG|44.0|ShoalLogger|_ThreadID=12;_ThreadName=Thread-2;|MySystemAdvertisement(dump)=PEERID[10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3] CUSTOMTAGS[GROUP_NAME:sqe-cluster, MEMBER_TYPE:CORE, START_TIME[8:54:29 AM PDT on May 31, 2012]], ENDPONTADDRESSES[[tcp://[fe80:0:0:0:223:8bff:fe64:d810%2]:4000, tcp://10.133.184.142:4000]], NAME:clustered_instance_3, OSNAME:Linux, OSVERSION:2.6.18-128.el5, OSARCH:i386, HWARCH:i386, HWVENDOR:Sun Microsystems Inc.|#] [#|2012-05-31T08:54:29.782-0700|INFO|44.0|org.glassfish.ha.store.spi.BackingStoreFactoryRegistry|_ThreadID=10;_ThreadName=Thread-2;|Registered org.glassfish.ha.store.adapter.cache.ShoalBackingStoreProxy for persistence-type = replicated in BackingStoreFactoryRegistry|#] [#|2012-05-31T08:54:29.764-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for MASTER_CHANGE_EVENT(before change analysis) are : 1: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 |#] [#|2012-05-31T08:54:29.803-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: MASTER_CHANGE_EVENT for member: clustered_instance_3 of group: sqe-cluster|#] [#|2012-05-31T08:54:30.271-0700|INFO|44.0|null|_ThreadID=10;_ThreadName=Thread-2;|this.makeModuleFor(org.glassfish.main.web.glue, null) returned OSGiModuleImpl:: Bundle = [org.glassfish.main.web.glue [108]], State = [READY]|#] [#|2012-05-31T08:54:30.272-0700|INFO|44.0|null|_ThreadID=10;_ThreadName=Thread-2;|this.makeModuleFor(org.glassfish.web.javax.servlet.jsp, null) returned OSGiModuleImpl:: Bundle = [org.glassfish.web.javax.servlet.jsp [160]], State = [READY]|#] [#|2012-05-31T08:54:30.368-0700|INFO|44.0|null|_ThreadID=10;_ThreadName=Thread-2;|this.makeModuleFor(org.glassfish.web.javax.el, null) returned OSGiModuleImpl:: Bundle = [org.glassfish.web.javax.el [13]], State = [READY]|#] [#|2012-05-31T08:54:30.369-0700|INFO|44.0|global|_ThreadID=10;_ThreadName=Thread-2;|Snifer org.glassfish.web.sniffer.WebSniffer@1ccb7 set up following modules: [OSGiModuleImpl:: Bundle = [org.glassfish.main.web.glue [108]], State = [READY], OSGiModuleImpl:: Bundle = [org.glassfish.web.javax.servlet.jsp [160]], State = [READY], OSGiModuleImpl:: Bundle = [org.glassfish.web.javax.el [13]], State = [READY]]|#] [#|2012-05-31T08:54:30.403-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1093: adding GroupLeadershipNotification signal leadermember: clustered_instance_3 of group: sqe-cluster|#] [#|2012-05-31T08:54:30.404-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for MASTER_CHANGE_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:54:30.404-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: MASTER_CHANGE_EVENT for member: server of group: sqe-cluster|#] [#|2012-05-31T08:54:30.404-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1093: adding GroupLeadershipNotification signal leadermember: server of group: sqe-cluster|#] [#|2012-05-31T08:54:30.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1024: Adding Join member: clustered_instance_1 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:30.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1024: Adding Join member: clustered_instance_2 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:30.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for ADD_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:54:30.407-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: ADD_EVENT for member: clustered_instance_3 of group: sqe-cluster|#] [#|2012-05-31T08:54:30.407-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1024: Adding Join member: clustered_instance_3 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:30.813-0700|INFO|44.0|javax.enterprise.system.core.transaction.com.sun.jts.CosTransactions|_ThreadID=10;_ThreadName=Thread-2;|JTS5014: Recoverable JTS instance, serverId = [100]|#] [#|2012-05-31T08:54:30.992-0700|INFO|44.0|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=10;_ThreadName=Thread-2;|Grizzly Framework 2.2.9 started in: 4ms - bound to [/0.0.0.0:58,080]|#] [#|2012-05-31T08:54:31.023-0700|INFO|44.0|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=10;_ThreadName=Thread-2;|Grizzly Framework 2.2.9 started in: 4ms - bound to [/0.0.0.0:58,181]|#] [#|2012-05-31T08:54:31.057-0700|INFO|44.0|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=10;_ThreadName=Thread-2;|Grizzly Framework 2.2.9 started in: 4ms - bound to [/0.0.0.0:54,848]|#] [#|2012-05-31T08:54:31.628-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0169: Created HTTP listener [http-listener-1] on host/port [0.0.0.0:58080]|#] [#|2012-05-31T08:54:31.648-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0169: Created HTTP listener [http-listener-2] on host/port [0.0.0.0:58181]|#] [#|2012-05-31T08:54:31.654-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0169: Created HTTP listener [admin-listener] on host/port [0.0.0.0:54848]|#] [#|2012-05-31T08:54:31.734-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0171: Created virtual server [server]|#] [#|2012-05-31T08:54:31.739-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0171: Created virtual server [__asadmin]|#] [#|2012-05-31T08:54:32.631-0700|INFO|44.0|org.apache.catalina.realm.JAASRealm|_ThreadID=10;_ThreadName=Thread-2;|Setting JAAS app name glassfish-web|#] [#|2012-05-31T08:54:32.632-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=10;_ThreadName=Thread-2;|WEB0172: Virtual server [server] loaded default web module []|#] [#|2012-05-31T08:54:32.800-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:54:32.800-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: JOINED_AND_READY_EVENT for member: clustered_instance_1 of group: sqe-cluster|#] [#|2012-05-31T08:54:32.800-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1025: Adding Joined And Ready member: clustered_instance_1 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:32.968-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:54:32.969-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: JOINED_AND_READY_EVENT for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:54:32.969-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1025: Adding Joined And Ready member: clustered_instance_2 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:34.477-0700|INFO|44.0|javax.enterprise.system.core.org.glassfish.kernel.javaee|_ThreadID=10;_ThreadName=Thread-2;|core.start_container_done|#] [#|2012-05-31T08:54:34.683-0700|INFO|44.0|javax.enterprise.system.core.com.sun.enterprise.v3.server|_ThreadID=10;_ThreadName=Thread-2;|Oracle GlassFish Server 4.0 (39) startup time : Felix (5,926ms), startup services(5,894ms), total(11,820ms)|#] [#|2012-05-31T08:54:34.684-0700|INFO|44.0|ShoalLogger|_ThreadID=10;_ThreadName=Thread-2;|GMS1099: GMS:Reporting Joined and Ready state to group: sqe-cluster|#] [#|2012-05-31T08:54:34.798-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:54:34.798-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: JOINED_AND_READY_EVENT for member: clustered_instance_3 of group: sqe-cluster|#] [#|2012-05-31T08:54:34.798-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1025: Adding Joined And Ready member: clustered_instance_3 group: sqe-cluster StartupState: GROUP_STARTUP |#] [#|2012-05-31T08:54:35.676-0700|INFO|44.0|javax.enterprise.system.core.com.sun.enterprise.v3.services.impl|_ThreadID=10;_ThreadName=Thread-2;|Grizzly Framework 2.2.9 started in: 671ms - bound to [/0.0.0.0:58,686]|#] [#|2012-05-31T08:54:36.395-0700|INFO|44.0|javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.inbound|_ThreadID=13;_ThreadName=Thread-2;|Recovery of Inbound Transactions started.|#] [#|2012-05-31T08:54:36.869-0700|INFO|44.0|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=14;_ThreadName=Thread-2;|/export/hudson/workspace/sherry-lc-dbg/glassfish3/glassfish/nodes/localhost-sqe-domain/clustered_instance_3/autodeploy/bundles does not exist, please create it.|#] [#|2012-05-31T08:54:37.607-0700|INFO|44.0|org.hibernate.validator.util.Version|_ThreadID=13;_ThreadName=Thread-2;|Hibernate Validator 4.1.0.Final|#] [#|2012-05-31T08:54:37.645-0700|INFO|44.0|org.hibernate.validator.engine.resolver.DefaultTraversableResolver|_ThreadID=13;_ThreadName=Thread-2;|Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.|#] [#|2012-05-31T08:54:38.226-0700|INFO|44.0|javax.enterprise.system.jmx.org.glassfish.admin.mbeanserver|_ThreadID=15;_ThreadName=Thread-2;|JMX005: JMXStartupService had Started JMXConnector on JMXService URL service:jmx:rmi://apg-sqe2:57676/jndi/rmi://apg-sqe2:57676/jmxrmi|#] [#|2012-05-31T08:54:38.371-0700|SEVERE|44.0|javax.enterprise.resource.jta.com.sun.enterprise.transaction|_ThreadID=13;_ThreadName=Thread-2;|DTX5016:Error in XA recovery. See logs for more details java.lang.NullPointerException at com.sun.enterprise.resource.recovery.JdbcRecoveryResourceHandler.loadXAResourcesAndItsConnections(JdbcRecoveryResourceHandler.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:240) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverXAResources(ResourceRecoveryManagerImpl.java:337) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.postConstruct(ResourceRecoveryManagerImpl.java:108) at com.sun.hk2.component.AbstractCreatorImpl.inject(AbstractCreatorImpl.java:132) at com.sun.hk2.component.ConstructorCreator.initialize(ConstructorCreator.java:117) at com.sun.hk2.component.AbstractCreatorImpl.get(AbstractCreatorImpl.java:84) at com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:67) at com.sun.hk2.component.EventPublishingInhabitant.get(EventPublishingInhabitant.java:141) at com.sun.hk2.component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:135) at org.jvnet.hk2.component.Habitat.getByContract(Habitat.java:1428) at com.sun.jts.jta.TransactionServiceProperties$RecoveryHelperThread.run(TransactionServiceProperties.java:360) |#] [#|2012-05-31T08:54:41.209-0700|INFO|44.0|org.hibernate.validator.engine.resolver.DefaultTraversableResolver|_ThreadID=16;_ThreadName=Thread-2;|Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.|#] [#|2012-05-31T08:54:52.619-0700|INFO|44.0|global|_ThreadID=16;_ThreadName=Thread-2;|Snifer org.glassfish.ejb.EjbSniffer@189c0 set up following modules: []|#] [#|2012-05-31T08:54:53.062-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security|_ThreadID=16;_ThreadName=Thread-2;|SEC1002: Security Manager is OFF.|#] [#|2012-05-31T08:54:53.084-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security|_ThreadID=16;_ThreadName=Thread-2;|SEC1010: Entering Security Startup Service|#] [#|2012-05-31T08:54:53.085-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security|_ThreadID=16;_ThreadName=Thread-2;|SEC1143: Loading policy provider com.sun.enterprise.security.provider.PolicyWrapper.|#] [#|2012-05-31T08:54:53.163-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=16;_ThreadName=Thread-2;|SEC1115: Realm [admin-realm] of classtype [com.sun.enterprise.security.auth.realm.file.FileRealm] successfully created.|#] [#|2012-05-31T08:54:53.164-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=16;_ThreadName=Thread-2;|SEC1115: Realm [file] of classtype [com.sun.enterprise.security.auth.realm.file.FileRealm] successfully created.|#] [#|2012-05-31T08:54:53.195-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security.auth.realm|_ThreadID=16;_ThreadName=Thread-2;|SEC1115: Realm [certificate] of classtype [com.sun.enterprise.security.auth.realm.certificate.CertificateRealm] successfully created.|#] [#|2012-05-31T08:54:53.196-0700|INFO|44.0|javax.enterprise.system.core.security.com.sun.enterprise.security|_ThreadID=16;_ThreadName=Thread-2;|SEC1011: Security Service(s) Started Successfully|#] [#|2012-05-31T08:54:53.196-0700|INFO|44.0|global|_ThreadID=16;_ThreadName=Thread-2;|Snifer com.sun.enterprise.security.ee.SecuritySniffer@389278a3 set up following modules: null|#] [#|2012-05-31T08:54:54.280-0700|INFO|44.0|javax.enterprise.system.container.ejb.com.sun.ejb.containers|_ThreadID=16;_ThreadName=Thread-2;|EJB5181:Portable JNDI names for EJB MyBean: [java:global/cliweb5/MyBean, java:global/cliweb5/MyBean!txn.recovery.cliweb5.MyBean]|#] [#|2012-05-31T08:54:54.478-0700|INFO|44.0|javax.enterprise.system.container.web.com.sun.enterprise.web|_ThreadID=16;_ThreadName=Thread-2;|WEB0671: Loading application [cliweb5] at [/cliweb5]|#] [#|2012-05-31T08:54:54.614-0700|INFO|44.0|javax.enterprise.system.tools.admin.org.glassfish.deployment.admin|_ThreadID=16;_ThreadName=Thread-2;|cliweb5 was successfully deployed in 2,039 milliseconds.|#] [#|2012-05-31T08:55:47.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for IN_DOUBT_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9158:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:55:47.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: IN_DOUBT_EVENT for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:55:47.406-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1007: Received FailureSuspectedEvent for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:55:47.407-0700|INFO|44.0|ShoalLogger|_ThreadID=17;_ThreadName=Thread-2;|GMS1005: Sending FailureSuspectedSignals to registered Actions. member: clustered_instance_2 ...|#] [#|2012-05-31T08:55:49.412-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for FAILURE_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 3: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:55:49.412-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: FAILURE_EVENT for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:55:49.413-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1019: member: clustered_instance_2 of group: sqe-cluster has failed|#] [#|2012-05-31T08:55:49.415-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1102: Appointed Recovery Server: clustered_instance_3 for failed member: clustered_instance_2 for group: sqe-cluster|#] [#|2012-05-31T08:55:49.419-0700|INFO|44.0|ShoalLogger|_ThreadID=17;_ThreadName=Thread-2;|GMS1004: Sending FailureRecoveryNotification to component TRANSACTION-RECOVERY-SERVICE member: clustered_instance_2|#] [#|2012-05-31T08:55:49.421-0700|INFO|44.0|ShoalLogger|_ThreadID=17;_ThreadName=Thread-2;|GMS1003: Sending FailureNotificationSignals to registered Actions. member: clustered_instance_2 ...|#] [#|2012-05-31T08:55:49.422-0700|INFO|44.0|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=18;_ThreadName=Thread-2;|[GMSCallBack] failure recovery signal: com.sun.enterprise.ee.cms.impl.common.FailureRecoverySignalImpl@1c0bd17|#] [#|2012-05-31T08:56:49.433-0700|INFO|44.0|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=18;_ThreadName=Thread-2;|[GMSCallBack] Recovering for instance: clustered_instance_2 logdir: clustered_instance_2|#] [#|2012-05-31T08:56:49.438-0700|INFO|44.0|javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.inbound|_ThreadID=18;_ThreadName=Thread-2;|Recovery of Inbound Transactions started.|#] [#|2012-05-31T08:56:49.500-0700|INFO|44.0|org.hibernate.validator.engine.resolver.DefaultTraversableResolver|_ThreadID=18;_ThreadName=Thread-2;|Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.|#] [#|2012-05-31T08:56:49.576-0700|WARNING|44.0|javax.enterprise.resource.jta.com.sun.enterprise.transaction|_ThreadID=18;_ThreadName=Thread-2;|DTX5016:Error in XA recovery. See logs for more details java.lang.NullPointerException at com.sun.enterprise.resource.recovery.JdbcRecoveryResourceHandler.loadXAResourcesAndItsConnections(JdbcRecoveryResourceHandler.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverIncompleteTx(ResourceRecoveryManagerImpl.java:149) at com.sun.enterprise.transaction.jts.recovery.GMSCallBack.doRecovery(GMSCallBack.java:239) at com.sun.enterprise.transaction.jts.recovery.GMSCallBack.processNotification(GMSCallBack.java:158) at com.sun.enterprise.ee.cms.impl.client.FailureRecoveryActionImpl.notifyListeners(FailureRecoveryActionImpl.java:100) at com.sun.enterprise.ee.cms.impl.client.FailureRecoveryActionImpl.consumeSignal(FailureRecoveryActionImpl.java:82) at com.sun.enterprise.ee.cms.impl.common.Router$CallableAction.call(Router.java:687) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) |#] [#|2012-05-31T08:56:49.577-0700|WARNING|44.0|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=18;_ThreadName=Thread-2;|JTS5074: Error while doing the transaction recovery. java.lang.NullPointerException at com.sun.enterprise.resource.recovery.JdbcRecoveryResourceHandler.loadXAResourcesAndItsConnections(JdbcRecoveryResourceHandler.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.getAllRecoverableResources(ResourceRecoveryManagerImpl.java:212) at com.sun.enterprise.transaction.jts.ResourceRecoveryManagerImpl.recoverIncompleteTx(ResourceRecoveryManagerImpl.java:149) at com.sun.enterprise.transaction.jts.recovery.GMSCallBack.doRecovery(GMSCallBack.java:239) at com.sun.enterprise.transaction.jts.recovery.GMSCallBack.processNotification(GMSCallBack.java:158) at com.sun.enterprise.ee.cms.impl.client.FailureRecoveryActionImpl.notifyListeners(FailureRecoveryActionImpl.java:100) at com.sun.enterprise.ee.cms.impl.client.FailureRecoveryActionImpl.consumeSignal(FailureRecoveryActionImpl.java:82) at com.sun.enterprise.ee.cms.impl.common.Router$CallableAction.call(Router.java:687) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) |#] [#|2012-05-31T08:56:49.577-0700|INFO|44.0|javax.enterprise.system.core.transaction.com.sun.jts.jta|_ThreadID=18;_ThreadName=Thread-2;|[GMSCallBack] Finished recovery for instance: clustered_instance_2|#] [#|2012-05-31T08:58:47.771-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for ADD_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9128:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:58:47.772-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: ADD_EVENT for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:58:47.772-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1024: Adding Join member: clustered_instance_2 group: sqe-cluster StartupState: INSTANCE_STARTUP |#] [#|2012-05-31T08:58:52.767-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1092: GMS View Change Received for group: sqe-cluster : Members in view for JOINED_AND_READY_EVENT(before change analysis) are : 1: MemberId: clustered_instance_1, MemberType: CORE, Address: 10.133.184.142:9116:228.9.71.122:14955:sqe-cluster:clustered_instance_1 2: MemberId: clustered_instance_2, MemberType: CORE, Address: 10.133.184.142:9128:228.9.71.122:14955:sqe-cluster:clustered_instance_2 3: MemberId: clustered_instance_3, MemberType: CORE, Address: 10.133.184.142:9103:228.9.71.122:14955:sqe-cluster:clustered_instance_3 4: MemberId: server, MemberType: SPECTATOR, Address: 10.133.184.142:9156:228.9.71.122:14955:sqe-cluster:server |#] [#|2012-05-31T08:58:52.768-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1016: Analyzing new membership snapshot received as part of event: JOINED_AND_READY_EVENT for member: clustered_instance_2 of group: sqe-cluster|#] [#|2012-05-31T08:58:52.768-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1025: Adding Joined And Ready member: clustered_instance_2 group: sqe-cluster StartupState: INSTANCE_STARTUP |#] [#|2012-05-31T09:01:02.010-0700|INFO|44.0|ShoalLogger|_ThreadID=19;_ThreadName=Thread-2;|GMS1015: Received Group Shutting down message from member: server of group: sqe-cluster|#] [#|2012-05-31T09:01:02.124-0700|INFO|44.0|javax.enterprise.system.tools.admin.com.sun.enterprise.v3.admin.cluster|_ThreadID=20;_ThreadName=Thread-2;|Server shutdown initiated|#] [#|2012-05-31T09:01:02.125-0700|INFO|44.0|javax.enterprise.system.std.com.sun.enterprise.server.logging|_ThreadID=20;_ThreadName=Thread-2;|FileMonitoring shutdown|#] [#|2012-05-31T09:01:02.125-0700|INFO|44.0|javax.org.glassfish.gms.org.glassfish.gms|_ThreadID=20;_ThreadName=Thread-2;|GMSAD1008: GMSAdapter for member: clustered_instance_3 group: sqe-cluster received GlassfishEventType: prepare_shutdown|#] [#|2012-05-31T09:01:02.126-0700|INFO|44.0|ShoalLogger|_ThreadID=20;_ThreadName=Thread-2;|GMS1096: member: clustered_instance_3 is leaving group: sqe-cluster|#] [#|2012-05-31T09:01:02.126-0700|INFO|44.0|ShoalLogger|_ThreadID=20;_ThreadName=Thread-2;|GMS1009: Leaving GMS group: sqe-cluster with shutdown type set to GroupShutdown|#] [#|2012-05-31T09:01:02.127-0700|CONFIG|44.0|ShoalLogger|_ThreadID=21;_ThreadName=Thread-2;|GMS1065: Completed processing outstanding master node messages for member: clustered_instance_3 group: sqe-cluster outstandingMessages to process: 0|#] [#|2012-05-31T09:01:02.127-0700|INFO|44.0|ShoalLogger|_ThreadID=22;_ThreadName=Thread-2;|GMS1110: Thread IP Multicast Listener for /228.9.71.122:14955 has completed.|#] [#|2012-05-31T09:01:02.128-0700|INFO|44.0|ShoalLogger.monitor|_ThreadID=20;_ThreadName=Thread-2;|BlockingIOMulicastSender monitoring stats: received: 710 core poolsize:10 largest pool size:10 task count:710 max queue size:0 rejected execution:0|#] [#|2012-05-31T09:01:02.146-0700|INFO|44.0|ShoalLogger|_ThreadID=17;_ThreadName=Thread-2;|GMS1107: SignalHandler task named GMS SignalHandler for Group-sqe-cluster thread exiting|#] [#|2012-05-31T09:01:02.146-0700|INFO|44.0|ShoalLogger|_ThreadID=11;_ThreadName=Thread-2;|GMS1091: View Window event processing thread for group: sqe-cluster terminated normally|#] [#|2012-05-31T09:01:02.147-0700|INFO|44.0|ShoalLogger.monitor|_ThreadID=20;_ThreadName=Thread-2;|GMS1115: router signal queue high water mark: 0 signal queue capacity: 600|#] [#|2012-05-31T09:01:02.156-0700|INFO|44.0|ShoalLogger|_ThreadID=19;_ThreadName=Thread-2;|GMS1088: MessageWindow thread for group: sqe-cluster terminated due to shutdown notification|#] [#|2012-05-31T09:01:02.161-0700|INFO|44.0|javax.enterprise.system.jmx.org.glassfish.admin.mbeanserver|_ThreadID=20;_ThreadName=Thread-2;|JMX002: JMXStartupService: Stopped JMXConnectorServer: null|#] [#|2012-05-31T09:01:02.162-0700|INFO|44.0|javax.enterprise.system.jmx.org.glassfish.admin.mbeanserver|_ThreadID=20;_ThreadName=Thread-2;|JMX001: JMXStartupService and JMXConnectors have been shut down.|#] [#|2012-05-31T09:01:02.197-0700|INFO|44.0|javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.service|_ThreadID=23;_ThreadName=Thread-2;|RAR7094: __xa_jdbc_ra shutdown successful.|#] [#|2012-05-31T09:01:02.199-0700|INFO|44.0|javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.service|_ThreadID=24;_ThreadName=Thread-2;|RAR7094: __ds_jdbc_ra shutdown successful.|#]