dev@glassfish.java.net

Re: Instability of QuickLook tests

From: Jane Young <Jane.Young_at_Sun.COM>
Date: Tue, 06 Oct 2009 11:33:21 -0700

Hi Lloyd,

I agree that the amx-tests failure is detecting valid bugs introduced to
v3 workspace. However, one thing I don't understand. Why is it that
when I run QL locally it passes in the first run and on Hudon it shows
failures and occasionally it passes. If it is a failure, it should
always fail.

I still don't understand why 2nd QL run always fail in amx-tests.

Jane


Lloyd Chambers wrote:
> My comments on these failures below, inline.
>
> —Lloyd
>
> On Oct 4, 2009, at 6:19 PM, Ming Zhang wrote:
>
>> Sreenivas Munnangi wrote:
>>> Ming Zhang wrote:
>>>> Jane,
>>>>
>>>> I have checked in one more test to delete http-listener "ls123452".
>>>> Now admin tests can be run repeatedly without errors and the
>>>> domain.xml gets cleaned up after each admin test run.
>>>>
>>>> Lloyd,
>>>> the AMX tests still failed in the 2nd QL run
>>> How many tests failed, if possible could you copy and paste the
>>> failures from ql output.
>>>> (iterateAllSanityCheck, testAMXComplianceMonitorFailureCount) after
>>>> my change for admin tests. Is this due to the configuration changes
>>>> in domain.xml after admin GUI loaded? Is it reasonable to expect
>>>> AMX tests will accommodate the configurations of admin GUI in
>>>> domain.xml?
>>>>
>>>> Thanks,
>>>> Ming
>> The runtestng.output is attached:
>>
>> [echo] =============Starting TestNG functional tests from
>> testng.xml ============
>> [testng] [Parser] Running:
>> [testng] /export/home/ming/v3/quicklook/testng.xml
>> [testng]
>> [testng] PASSED: createJDBCPool
>> [testng] PASSED: pingJDBCPool
>> [testng] PASSED: deleteJDBCPool
>> [testng]
>> [testng] ===============================================
>> [testng] admincli_tests
>> [testng] Tests run: 3, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] Monitoring at HIGH for ThreadPool
>> [testng] Monitoring at HIGH for JdbcConnectionPool
>> [testng] Monitoring at HIGH for Orb
>> [testng] Monitoring at HIGH for ConnectorService
>> [testng] Monitoring at HIGH for Security
>> [testng] Monitoring at HIGH for JmsService
>> [testng] Monitoring at HIGH for WebServicesContainer
>> [testng] Monitoring at HIGH for Jvm
>> [testng] Monitoring at HIGH for Jpa
>> [testng] Monitoring at HIGH for Jersey
>> [testng] Monitoring at HIGH for ConnectorConnectionPool
>> [testng] Monitoring at HIGH for EjbContainer
>> [testng] Monitoring at HIGH for WebContainer
>> [testng] Monitoring at HIGH for TransactionService
>> [testng] Monitoring at HIGH for HttpService
>> [testng]
>> [testng] PROBLEMS:
>> [testng] Error invoking getParent() on
>> amx:pp=/mon/server-mon[server],type=jdbc-connection-pool-mon,name=resources/__TimerPool
>> = java.lang.reflect.InvocationTargetException
>
> This root cause is needed here, but if getParent() is failing because
> there is no parent, then this is a critical bug in this MBean,
> supplied by the monitoring infrastructure.
>
>> [testng] CHANGE COUNT: 10
>> [testng] PASSED: bootAMX
>> [testng] Core AMX tests
>> [testng] PASSED: testAmxPref
>> [testng] PASSED: connectorConnectionPoolTest
>> [testng] PASSED: createChildTest
>> [testng] PASSED: testConfigTools
>> [testng] PASSED: createProfilerTest
>> [testng] PASSED: testAMXConfigAttributeResolver
>> [testng] PASSED: testAMXConfigDefaultValues
>> [testng] PASSED: testForBogusConfigAnnotations
>> [testng] PASSED: testCreateProperties
>> [testng] PASSED: testCreateResource
>> [testng] PASSED: testCreateProperty
>> [testng] PASSED: testConnectorRuntimeAPIProvider
>> [testng] PASSED: testMonitoringRoot
>> [testng] PASSED: testDomainConfig
>> [testng] PASSED: testBulkAccess
>> [testng] PASSED: testExt
>> [testng] PASSED: testPropertyParent
>> [testng] PASSED: testTools
>> [testng] PASSED: testPathnames
>> [testng] PASSED: testSystemStatus
>> [testng] PASSED: testApplications
>> [testng] PASSED: testResources
>> [testng] PASSED: testChildGetterVariants
>> [testng] PASSED: testAllGenerically
>> [testng] PASSED: testRuntimeRoot
>> [testng] PASSED: testQuery
>> [testng] PASSED: testJ2EEDomain
>> [testng] PASSED: testSingletonOrNot
>> [testng] PASSED: testDomainRoot
>> [testng] PASSED: testSystemApplications
>> [testng] PASSED: testSystemInfo
>> [testng] PASSED: testSystemPropertyParent
>> [testng] PASSED: testConfigs
>> [testng] PASSED: testServerRuntime
>> [testng] PASSED: testLogging
>> [testng] PASSED: testServers
>> [testng] PASSED: testAutoConvert
>> [testng] PASSED: testForBogusAnnotations
>> [testng] FAILED: iterateAllSanityCheck
>
> This should be suppressed as of my Friday checkin. When was this run?
>
>> [testng] java.lang.RuntimeException:
>> javax.management.InstanceNotFoundException:
>> amx:pp=/mon/server-mon[server],type=bean-method-mon,name=slsbnicmt/AnnotatedEJB/bean-methods/persistEntity
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.ProxyFactory.getMBeanInfo(ProxyFactory.java:447)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.ProxyFactory.getProxy(ProxyFactory.java:482)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.ProxyFactory.toProxySet(ProxyFactory.java:627)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.AMXProxyHandler.autoConvert(AMXProxyHandler.java:653)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.AMXProxyHandler._invoke(AMXProxyHandler.java:740)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.AMXProxyHandler.invoke(AMXProxyHandler.java:518)
>> [testng] at $Proxy19.queryAll(Unknown Source)
>> [testng] at amxtest.AMXTestBase.getAllAMX(AMXTestBase.java:295)
>> [testng] at
>> amxtest.AMXCoreTests.iterateAllSanityCheck(AMXCoreTests.java:86)
>> [testng] Caused by: javax.management.InstanceNotFoundException:
>> amx:pp=/mon/server-mon[server],type=bean-method-mon,name=slsbnicmt/AnnotatedEJB/bean-methods/persistEntity
>> [testng] at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1094)
>> [testng] at
>> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1372)
>> [testng] at
>> com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:880)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1420)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl.getMBeanInfo(RMIConnectionImpl.java:864)
>> [testng] at
>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>> [testng] at sun.rmi.transport.Transport$1.run(Transport.java:159)
>> [testng] at java.security.AccessController.doPrivileged(Native Method)
>> [testng] at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>> [testng] at
>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>> [testng] at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>> [testng] at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>> [testng] at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> [testng] at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> [testng] at java.lang.Thread.run(Thread.java:619)
>> [testng] at
>> sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:255)
>> [testng] at
>> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
>> [testng] at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:142)
>> [testng] at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
>> [testng] at
>> javax.management.remote.rmi.RMIConnectionImpl_Stub.getMBeanInfo(Unknown
>> Source)
>> [testng] at
>> javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getMBeanInfo(RMIConnector.java:1037)
>> [testng] at
>> org.glassfish.admin.amx.core.proxy.ProxyFactory.getMBeanInfo(ProxyFactory.java:437)
>> [testng] ... 30 more
>> [testng] ... Removed 25 stack frames
>> [testng] FAILED: testAMXComplianceMonitorFailureCount
>> [testng] java.lang.AssertionError: Server indicates that there are
>> non-compliant AMX MBean validator failures, failure count = 1,
>> examine the server log for failures
>> [testng] at
>> amxtest.AMXCoreTests.testAMXComplianceMonitorFailureCount(AMXCoreTests.java:120)
>> [testng] ... Removed 22 stack frames
>
> This indicates non-compliant MBeans. I will see if I can make the
> message spell out which MBeans and what failed.
>
>> [testng]
>> [testng] ===============================================
>> [testng] amx_tests
>> [testng] Tests run: 41, Failures: 2, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] ***** Login page not found. Sleeping to allow app to
>> deploy (1 of 60)...
>> [testng] ***** Login page not found. Sleeping to allow app to
>> deploy (2 of 60)...
>> [testng] ***** Login page not found. Sleeping to allow app to
>> deploy (3 of 60)...
>> [testng] ***** Login page not found. Sleeping to allow app to
>> deploy (4 of 60)...
>> [testng] ***** Login page not found. Sleeping to allow app to
>> deploy (5 of 60)...
>> [testng] Oct 4, 2009 5:51:08 PM
>> org.apache.commons.httpclient.HttpMethodDirector isRedirectNeeded
>> [testng] INFO: Redirect requested but followRedirects is disabled
>> [testng] PASSED: testDeployedAppPage
>> [testng] PASSED: testCommonTasks
>> [testng] PASSED: testHelpPage
>> [testng] PASSED: testRealmsList
>> [testng]
>> [testng] ===============================================
>> [testng] adminconsole_tests
>> [testng] Tests run: 4, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: staticHTMLPageTest
>> [testng] PASSED: testServerRunning
>> [testng] PASSED: simpleServletTest
>> [testng] PASSED: simpleJSPTestPage
>> [testng]
>> [testng] ===============================================
>> [testng] web_jspservlet_war
>> [testng] Tests run: 4, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: jsfIndexPageBasicTest
>> [testng] PASSED: jsfAppDeployedFirstPagetest
>> [testng]
>> [testng] ===============================================
>> [testng] jsf_hello_world
>> [testng] Tests run: 2, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] Injection2 matched.
>> [testng] PASSED: injectedValuesTestPage
>> [testng]
>> [testng] ===============================================
>> [testng] jsf_injection
>> [testng] Tests run: 1, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: testNoLeak
>> [testng] PASSED: testUserTx
>> [testng]
>> [testng] ===============================================
>> [testng] jdbc_user_tx
>> [testng] Tests run: 2, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: testJpaPersistWithInjectEMF
>> [testng] PASSED: testJpaLazyLoadingByQuery
>> [testng]
>> [testng] ===============================================
>> [testng] jpa_inject_emf
>> [testng] Tests run: 2, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: staticHTMLPageTest
>> [testng] PASSED: simpleServletTest
>> [testng] PASSED: simpleJSPTestPage
>> [testng]
>> [testng] ===============================================
>> [testng] security_hello_jsp
>> [testng] Tests run: 3, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: testAuthGroupMappedUser
>> [testng] PASSED: testAuthNotValidPassword
>> [testng] PASSED: testAuthRoleMappedUser
>> [testng] PASSED: testAuthNotAuthorizedUser
>> [testng]
>> [testng] ===============================================
>> [testng] security_hello_basicauth
>> [testng] Tests run: 4, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: SetName
>> [testng] PASSED: EJBInject
>> [testng] PASSED: GetName
>> [testng] PASSED: JpaPersist
>> [testng] PASSED: JpaRemove
>> [testng] PASSED: JpaVerify
>> [testng]
>> [testng] ===============================================
>> [testng] ejb_stateful_cmt_ni_war
>> [testng] Tests run: 6, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: executeServlet
>> [testng]
>> [testng] ===============================================
>> [testng] bv_servlet_simple
>> [testng] Tests run: 1, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: appDeployedFirstPagetest
>> [testng]
>> [testng] ===============================================
>> [testng] webbeans_numberguess
>> [testng] Tests run: 1, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng] PASSED: createListener
>> [testng] PASSED: createPool
>> [testng] PASSED: createJoe
>> [testng] PASSED: ensureCreatedJoeExists
>> [testng] PASSED: deleteJoe
>> [testng] PASSED: ensureCreatedListenerExists
>> [testng] PASSED: deleteListener
>> [testng] PASSED: ensureDeletedListenerDoesNotExist
>> [testng] PASSED: createListenerWithOldParam
>> [testng] PASSED: deleteListener2
>> [testng] PASSED: pingPool
>> [testng] PASSED: deletedJoeDoesNotExist
>> [testng] PASSED: ensureCreatedPoolExists
>> [testng] PASSED: deletePool
>> [testng] PASSED: deletedPoolDoesNotExist
>> [testng]
>> [testng] ===============================================
>> [testng] asadmin_tests
>> [testng] Tests run: 15, Failures: 0, Skips: 0
>> [testng] ===============================================
>> [testng]
>> [testng]
>> [testng] ===============================================
>> [testng] QuickLookTests
>> [testng] Total tests run: 89, Failures: 2, Skips: 0
>> [testng] ===============================================
>> [testng]
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> <mailto:dev-unsubscribe_at_glassfish.dev.java.net>
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>> <mailto:dev-help_at_glassfish.dev.java.net>
>
> Lloyd Chambers
> lloyd.chambers_at_sun.com <mailto:lloyd.chambers_at_sun.com>
> GlassFish Team
>
>
>