Ming Zhang wrote:
> Jane,
>
> I have checked in one more test to delete http-listener "ls123452".
> Now admin tests can be run repeatedly without errors and the
> domain.xml gets cleaned up after each admin test run.
>
> Lloyd,
> the AMX tests still failed in the 2nd QL run
How many tests failed, if possible could you copy and paste the failures
from ql output.
> (iterateAllSanityCheck, testAMXComplianceMonitorFailureCount) after my
> change for admin tests. Is this due to the configuration changes in
> domain.xml after admin GUI loaded? Is it reasonable to expect AMX
> tests will accommodate the configurations of admin GUI in domain.xml?
>
> Thanks,
> Ming
>
> Jane Young wrote:
>
>> Ming,
>>
>> Recreating the domain will resolve the AMX test failures. Looks
>> like QL may not be cleaning-up domain.xml and resetting to the
>> original configuration.
>> I see that admin-gui configuration is added to domain.xml after
>> running QL. This is because admin-gui is getting invoked the first
>> time. There is a new http-listener, "ls123452". Should that be
>> removed? I'm not sure what other configurations are added and need
>> to be removed after running QL.
>>
>> Can you look into this?
>>
>>
>> Thanks,
>> Jane
>>
>>
>> Ming Zhang wrote:
>>
>>> I tried to reproduce the issue with the promoted b66
>>> (http://download.java.net/glassfish/v3/promoted/glassfish-v3-b66.zip)
>>> and here are what I found:
>>>
>>> * Run QL first time in web distribution profile and all passed.
>>> * Run QL again and the amx or admin tests start having errors
>>> * To isolate the problem, I deleted the b66 and unzip again. Run
>>> amx test individually multiple times and tests passed:
>>> o quicklook/amx % ant
>>> -Dglassfish.home=/export/home/ming/v3/b66/glassfishv3/glassfish
>>> all
>>> * Start domain and derby. Run admin tests individually. The
>>> tests passed first time and started failed when ran again.
>>> o quicklook/admin % ant
>>> -Dglassfish.home=/export/home/ming/v3/b66/glassfishv3/glassfish
>>> all
>>>
>>> The stack trace on server side is clean:
>>> http://javaweb.sfbay/~mzh777/v3/ql/b66/server.log
>>> But there are exceptions on client side. Can someone on admin team
>>> take a look for the error?
>>>
>>> Thanks,
>>> Ming
>>>
>>> To turn on the the verbose mode for testng
>>> quicklook/admin % svn diff
>>> Index: testng.xml
>>> ===================================================================
>>> --- testng.xml (revision 32234)
>>> +++ testng.xml (working copy)
>>> @@ -35,7 +35,7 @@
>>> -->
>>> <!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
>>>
>>> -<suite name="QuickLookTests" verbose="2" >
>>> +<suite name="QuickLookTests" verbose="4" >
>>>
>>> and here are the output on console:
>>>
>>> runtest-impl-xml:
>>> [echo] =============Starting TestNG test at ..//classes/test
>>> from testng.xml ============
>>> [testng] [Parser] Running:
>>> [testng] /export/home/ming/v3/quicklook/admin/testng.xml
>>>
>>> [testng] [RunInfo] Adding method selector:
>>> org.testng.internal.XmlMethodSelector_at_ef2c60 priority: 10
>>> [testng] [TestClass] Creating TestClass for [ClassImpl
>>> test.admin.JdbcConnectionPoolTests]
>>> [testng] [TestClass] Adding method
>>> test.admin.JdbcConnectionPoolTests.createPool() on TestClass class
>>> test.admin.JdbcConnectionPoolTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JdbcConnectionPoolTests.ensureCreatedPoolExists() on
>>> TestClass class test.admin.JdbcConnectionPoolTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JdbcConnectionPoolTests.deletedPoolDoesNotExist() on
>>> TestClass class test.admin.JdbcConnectionPoolTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JdbcConnectionPoolTests.deletePool() on TestClass class
>>> test.admin.JdbcConnectionPoolTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JdbcConnectionPoolTests.pingPool() on TestClass class
>>> test.admin.JdbcConnectionPoolTests
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.setUpEnvironment()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.setupEnvironment()
>>> [testng] [MethodInheritance]
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)
>>> DEPENDS ON
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [TestClass] Creating TestClass for [ClassImpl
>>> test.admin.JvmOptionTests]
>>> [testng] [TestClass] Adding method
>>> test.admin.JvmOptionTests.deletedJoeDoesNotExist() on TestClass
>>> class test.admin.JvmOptionTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JvmOptionTests.deleteJoe() on TestClass class
>>> test.admin.JvmOptionTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JvmOptionTests.ensureCreatedJoeExists() on TestClass
>>> class test.admin.JvmOptionTests
>>> [testng] [TestClass] Adding method
>>> test.admin.JvmOptionTests.createJoe() on TestClass class
>>> test.admin.JvmOptionTests
>>> [testng] [TestClass] Creating TestClass for [ClassImpl
>>> test.admin.WebtierTests]
>>> [testng] [TestClass] Adding method
>>> test.admin.WebtierTests.deleteListener() on TestClass class
>>> test.admin.WebtierTests
>>> [testng] [TestClass] Adding method
>>> test.admin.WebtierTests.ensureDeletedListenerDoesNotExist() on
>>> TestClass class test.admin.WebtierTests
>>> [testng] [TestClass] Adding method
>>> test.admin.WebtierTests.createListenerWithOldParam() on TestClass
>>> class test.admin.WebtierTests
>>> [testng] [TestClass] Adding method
>>> test.admin.WebtierTests.ensureCreatedListenerExists() on TestClass
>>> class test.admin.WebtierTests
>>> [testng] [TestClass] Adding method
>>> test.admin.WebtierTests.createListener() on TestClass class
>>> test.admin.WebtierTests
>>> [testng] [XmlMethodSelector] Including method test.admin.setup()
>>> [testng] [MethodInheritance] test.admin.WebtierTests.setup()
>>> DEPENDS ON
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [MethodInheritance]
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)
>>> DEPENDS ON
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [MethodInheritance] test.admin.WebtierTests.setup()
>>> DEPENDS ON
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)
>>> [testng] [XmlMethodSelector] Including method test.admin.createPool()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.ensureCreatedPoolExists()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.deletedPoolDoesNotExist()
>>> [testng] [XmlMethodSelector] Including method test.admin.deletePool()
>>> [testng] [XmlMethodSelector] Including method test.admin.pingPool()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.deletedJoeDoesNotExist()
>>> [testng] [XmlMethodSelector] Including method test.admin.deleteJoe()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.ensureCreatedJoeExists()
>>> [testng] [XmlMethodSelector] Including method test.admin.createJoe()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.deleteListener()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.ensureDeletedListenerDoesNotExist()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.createListenerWithOldParam()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.ensureCreatedListenerExists()
>>> [testng] [XmlMethodSelector] Including method
>>> test.admin.createListener()
>>> [testng] [SuiteRunner] Created 1 TestRunners
>>> [testng] [TestRunner] Running test asadmin_tests on 3 classes,
>>> included groups:[] excluded groups:[]
>>> [testng] [TestClass]
>>> [testng] ======
>>> [testng] TESTCLASS: test.admin.JdbcConnectionPoolTests
>>> [testng] [TestClass] BeforeClass :
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [TestClass] BeforeClass :
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)
>>> [testng] [TestClass] Test :
>>> test.admin.JdbcConnectionPoolTests.createPool()
>>> [testng] [TestClass] Test :
>>> test.admin.JdbcConnectionPoolTests.ensureCreatedPoolExists()
>>> [testng] [TestClass] Test :
>>> test.admin.JdbcConnectionPoolTests.deletedPoolDoesNotExist()
>>> [testng] [TestClass] Test :
>>> test.admin.JdbcConnectionPoolTests.deletePool()
>>> [testng] [TestClass] Test :
>>> test.admin.JdbcConnectionPoolTests.pingPool()
>>> [testng] [TestClass]
>>> [testng] ======
>>>
>>> [testng] [TestClass]
>>> [testng] ======
>>> [testng] TESTCLASS: test.admin.JvmOptionTests
>>> [testng] [TestClass] BeforeClass :
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [TestClass] Test :
>>> test.admin.JvmOptionTests.deletedJoeDoesNotExist()
>>> [testng] [TestClass] Test :
>>> test.admin.JvmOptionTests.deleteJoe()
>>> [testng] [TestClass] Test :
>>> test.admin.JvmOptionTests.ensureCreatedJoeExists()
>>> [testng] [TestClass] Test :
>>> test.admin.JvmOptionTests.createJoe()
>>> [testng] [TestClass]
>>> [testng] ======
>>>
>>> [testng] [TestClass]
>>> [testng] ======
>>> [testng] TESTCLASS: test.admin.WebtierTests
>>> [testng] [TestClass] BeforeClass :
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [TestClass] BeforeClass : test.admin.WebtierTests.setup()
>>> [testng] [TestClass] Test :
>>> test.admin.WebtierTests.deleteListener()
>>> [testng] [TestClass] Test :
>>> test.admin.WebtierTests.ensureDeletedListenerDoesNotExist()
>>> [testng] [TestClass] Test :
>>> test.admin.WebtierTests.createListenerWithOldParam()
>>> [testng] [TestClass] Test :
>>> test.admin.WebtierTests.ensureCreatedListenerExists()
>>> [testng] [TestClass] Test :
>>> test.admin.WebtierTests.createListener()
>>> [testng] [TestClass]
>>> [testng] ======
>>>
>>> [testng] [TestRunner] WILL BE RUN IN RANDOM ORDER:
>>> [testng] [TestRunner]
>>> test.admin.WebtierTests.createListenerWithOldParam()
>>> [testng] [TestRunner] WILL BE RUN SEQUENTIALLY:
>>> [testng] [TestRunner] test.admin.WebtierTests.createListener()
>>> [testng] [TestRunner]
>>> test.admin.JdbcConnectionPoolTests.createPool()
>>> [testng] [TestRunner] test.admin.JvmOptionTests.createJoe()
>>> [testng] [TestRunner]
>>> test.admin.JvmOptionTests.ensureCreatedJoeExists()
>>> [testng] [TestRunner] test.admin.JvmOptionTests.deleteJoe()
>>> [testng] [TestRunner]
>>> test.admin.WebtierTests.ensureCreatedListenerExists()
>>> [testng] [TestRunner] test.admin.WebtierTests.deleteListener()
>>> [testng] [TestRunner]
>>> test.admin.WebtierTests.ensureDeletedListenerDoesNotExist()
>>> [testng] [TestRunner] test.admin.JdbcConnectionPoolTests.pingPool()
>>> [testng] [TestRunner]
>>> test.admin.JvmOptionTests.deletedJoeDoesNotExist()
>>> [testng] [TestRunner]
>>> test.admin.JdbcConnectionPoolTests.ensureCreatedPoolExists()
>>> [testng] [TestRunner]
>>> test.admin.JdbcConnectionPoolTests.deletePool()
>>> [testng] [TestRunner]
>>> test.admin.JdbcConnectionPoolTests.deletedPoolDoesNotExist()
>>> [testng] [TestRunner] ====
>>> [testng] [TestRunner] ===
>>> [testng] [TestRunner] Found 2 applicable methods
>>> [testng] [Invoker 20469344] Invoking @BeforeClass
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [Invoker 20469344] Invoking @BeforeClass
>>> test.admin.WebtierTests.setup()
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.WebtierTests.createListener
>>> [testng] [Invoker 20469344] Invoking @BeforeClass
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [Invoker 20469344] Invoking @BeforeClass
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JdbcConnectionPoolTests.createPool
>>> [testng] [Invoker 20469344] Invoking @BeforeClass
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JvmOptionTests.createJoe
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JvmOptionTests.ensureCreatedJoeExists
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JvmOptionTests.deleteJoe
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.WebtierTests.ensureCreatedListenerExists
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.WebtierTests.deleteListener
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JdbcConnectionPoolTests.pingPool
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JvmOptionTests.deletedJoeDoesNotExist
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JdbcConnectionPoolTests.ensureCreatedPoolExists
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JdbcConnectionPoolTests.deletePool
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.JdbcConnectionPoolTests.deletedPoolDoesNotExist
>>> [testng] [Invoker 20469344] Invoking
>>> test.admin.WebtierTests.createListenerWithOldParam
>>>
>>> [testng] *********** INVOKED METHODS
>>>
>>> [testng]
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)http://localhost:4848/__asadmin
>>> admin 7408068
>>> [testng] test.admin.WebtierTests.setup() 7408068
>>> [testng] test.admin.WebtierTests.createListener() 7408068
>>> [testng]
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)http://localhost:4848/__asadmin
>>> admin 24744797
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.setupEnvironment(java.lang.String)resources/resources.xml
>>> 24744797
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.createPool() 24744797
>>> [testng]
>>> test.admin.BaseAsadminTest.setUpEnvironment(java.lang.String,
>>> java.lang.String, java.lang.String)http://localhost:4848/__asadmin
>>> admin 9949215
>>> [testng] test.admin.JvmOptionTests.createJoe() 9949215
>>> [testng]
>>> test.admin.JvmOptionTests.ensureCreatedJoeExists() 9949215
>>> [testng] test.admin.JvmOptionTests.deleteJoe() 9949215
>>> [testng]
>>> test.admin.WebtierTests.ensureCreatedListenerExists() 7408068
>>> [testng] test.admin.WebtierTests.deleteListener() 7408068
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.pingPool() 24744797
>>> [testng]
>>> test.admin.JvmOptionTests.deletedJoeDoesNotExist() 9949215
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.ensureCreatedPoolExists() 24744797
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.deletePool() 24744797
>>> [testng]
>>> test.admin.JdbcConnectionPoolTests.deletedPoolDoesNotExist() 24744797
>>> [testng]
>>> test.admin.WebtierTests.createListenerWithOldParam() 7408068
>>>
>>> [testng] ***********
>>>
>>> [testng] Creating
>>> /export/home/ming/v3/quicklook/test-output/QuickLookTests/asadmin_tests.html
>>> [testng] PASSED: createListener
>>> [testng] PASSED: createPool
>>> [testng] PASSED: createJoe
>>> [testng] PASSED: ensureCreatedJoeExists
>>> [testng] PASSED: deleteJoe
>>> [testng] PASSED: ensureCreatedListenerExists
>>> [testng] PASSED: pingPool
>>> [testng] PASSED: deletedJoeDoesNotExist
>>> [testng] PASSED: ensureCreatedPoolExists
>>> [testng] PASSED: deletePool
>>> [testng] PASSED: deletedPoolDoesNotExist
>>> [testng] PASSED: createListenerWithOldParam
>>> [testng] FAILED: deleteListener
>>> [testng] java.lang.RuntimeException: null
>>> [testng] at
>>> test.admin.util.GeneralUtils.handleManifestFailure(GeneralUtils.java:124)
>>> [testng] at
>>> test.admin.WebtierTests.deleteListener(WebtierTests.java:101)
>>> [testng] ... Removed 22 stack frames
>>> [testng] SKIPPED: ensureDeletedListenerDoesNotExist
>>>
>>> [testng] ===============================================
>>> [testng] asadmin_tests
>>> [testng] Tests run: 14, Failures: 1, Skips: 1
>>> [testng] ===============================================
>>>
>>>
>>> [testng] ===============================================
>>> [testng] QuickLookTests
>>> [testng] Total tests run: 14, Failures: 1, Skips: 1
>>> [testng] ===============================================
>>>
>>> Ken Cavanaugh wrote:
>>>
>>>> Darani and I have been running the QLs on multiple machines in
>>>> order to
>>>> verify the functioning of ORB b030 (GFv3 is currently using ORB b029),
>>>> and every time we run the tests we get different results. I've had
>>>> the same
>>>> problems with trying to integrate Gmbal b016.
>>>>
>>>> The main tests that do this in the QL appear to be
>>>> iterateAllSanityCheck (an
>>>> AMX test), testAMXComplianceMonitorFailureCount, and deleteListener.
>>>> Over 4 QL runs, I've observed the following:
>>>>
>>>> * Run 1: passed with Gmbal b015
>>>> * Run 2: all 3 tests failed with Gmbal b016
>>>> * Run 3: with CORBA b030, testAMXComplianceMonitorFailureCount
>>>> failed
>>>> o The AMX compliance monitor complained about the following:
>>>> + Attribute 'Enabled' failed for
>>>> amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> + Attribute 'JvmOptions' failed for
>>>> amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> + Attribute 'NativeLibraryPath' failed for
>>>> amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> + Attribute 'Classpath' failed for
>>>> amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> + Attribute 'Property' failed for
>>>> amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> + amx:pp=/domain/configs/config[server-config]/java-config,type=profiler
>>>> General test failure in validateAMXConfig:
>>>> java.lang.NullPointerException: "null"
>>>> org.glassfish.admin.amx.core.AMXValidator.validateAMXConfig(AMXValidator.java:650)
>>>> org.glassfish.admin.amx.core.AMXValidator._validate(AMXValidator.java:618)
>>>> org.glassfish.admin.amx.core.AMXValidator.validate(AMXValidator.java:1213)
>>>> org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.doRun(ComplianceMonitor.java:235)
>>>> org.glassfish.admin.amx.impl.mbean.ComplianceMonitor$ValidatorThread.run(ComplianceMonitor.java:206)
>>>> * Run 4: I re-ran the same software as in run 3. This time all
>>>> 3 tests failed.
>>>> + There were several Gmbal failures in the log
>>>> (unregister failures). These were all caused by
>>>> InstanceNotFoundExceptions on a number of MBeans:
>>>> # amx:pp=/mon/server-mon[server],type=bean-method-mon,name=remoteview/HelloBean/bean-methods/hello
>>>> # amx:pp=/mon/server-mon[server],type=bean-method-mon,name=remoteview/HelloBean/bean-methods/asyncCancel-int
>>>> # amx:pp=/mon/server-mon[server],type=bean-method-mon,name=remoteview/HelloBean/bean-methods/asyncThrowException-java.lang.String
>>>> # amx:pp=/mon/server-mon[server],type=bean-method-mon,name=remoteview/HelloBean/bean-methods/throwException-java.lang.String
>>>> # amx:pp=/mon/server-mon[server],type=bean-method-mon,name=remoteview/HelloBean/bean-methods/asyncBlock-int
>>>> # etc. for a total of 153 MBeans: each
>>>> unregister failure happened twice in a row.
>>>>
>>>> Now, I initially assumed that the QL failures I observed in Run 2
>>>> were do to interactions between
>>>> a new Gmbal feature (root parent monitoring) and the rest of GFv3
>>>> monitoring. But the same tests
>>>> seem to fail almost randomly in other situations. Particularly
>>>> disturbing is the fact the Run 3 and Run 4
>>>> shared completely IDENTICAL app server code, and exhibited
>>>> different failures. I also highly doubt that
>>>> the ORB has anything to do with the observed (and unpredictable)
>>>> failures. Note that very similar failures
>>>> are seen both with Gmbal b016 and with CORBA b030. That
>>>> coincidence and the lack of repeatability lead
>>>> me to wonder about the stability of the QLs, and whether the
>>>> failures on Gmbal b016 have anything at all to do
>>>> with the recent Gmbal changes.
>>>>
>>>> This is a big problem. Darani and I have been unable to complete
>>>> any CORBA or Gmbal integrations recently,
>>>> because we keep seeing different failures. Darani had b30 passing
>>>> ALL tests at one point, then when she attempted
>>>> to verify one final time to complete the integration, similar
>>>> failures occurred.
>>>>
>>>> We need help to understand why we see this level of instability in
>>>> QL runs. Without fixing this, we
>>>> are unable to integrate either CORBA or Gmbal, and we are nearly
>>>> out of time for a number
>>>> of important fixes (many of which are ready to go) before hard code
>>>> freeze.
>>>>
>>>> Thanks,
>>>>
>>>> Ken.
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net For
>>>> additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>>
>>
>