Hi Sahoo,
Ideally, the failure in war deployment should fail the current test only
as you mentioned. But knowing all tests requiring war deployment will
fail at the same place any way, we will not see a big difference. The
current "failonerror" deployment will cause QL not to run the tests.
This is not an ideal implementation and I'll put this in my QL
improvement list. It'll also be good if we can add some deployment tests
in QL to isolate the deployment failure.
Thanks for providing a function list for a "reasonably good test
framework". I am working on improving QL and running single test of QL
will be in my top list since a lot of people mentioned that. I believe
TestNG has most of the function in the list and I will work on to fully
utilize the TestNG features. If you have any specific suggestion to
improve the framework, please let me know.
Thanks,
Ming
Sahoo wrote:
> Does it mean if any application does not get deployed, QL will not run
> any tests? Or, only that test does not run? I hope it's the latter as
> the former one is severely limiting. A reasonably good test framework
> should do the following:
>
> 1. Every test should be independent. So, one test failure should not
> affect another.
> 2. It should clearly indicate whether the failure was caused by :
> build/compilation failure, deployment failure, set up failure or
> actual execution failure. Simple error codes can be printed at the end
> to report this.
> 3. There should be an easy way to execute each of the above steps for
> every test.
> 4. It should be possible to run tests without compiling them every time.
> 5. Each test should have a separate log file (not server log, but
> client log) of its own so that the consolidated report is relatively
> small to look at.
> 6. It should be possible to configure the test framework to kill a
> test if it takes beyond a pre-configured time.
> 7. There should be a simple way to attach a debugger to the test.
>
> Tell me what QL satisfies from the above list.
>
> Thanks,
> Sahoo
>
> Ming Zhang wrote:
>> Hi Jerome,
>>
>> I have made the changes so that QL test will stop when deployment fails.
>> Thanks,
>> Ming
>>
>> Jerome Dochez wrote:
>>> most likely the deployment of that application did not go through.
>>> do you see an exception in the server log ?
>>>
>>> Ming, could you modify the QL so it stops when deployment of one of
>>> the tests fail. there is really no point in continuing the test run
>>> in such a case.
>>>
>>> jerome
>>>
>>> Carla Mott wrote:
>>>>
>>>> Under what conditions would I get a Connection refused error
>>>> below? It seems it is random. The web_jspservlet_war tests were
>>>> failing some times all and sometimes 2. When I updated my
>>>> quicklook workspace I see that the classloader_jsf test is failing
>>>> with the same exception and all the web_jspservlet_war tests are
>>>> now passing. There were no changes to the web_jspservlet_war tests
>>>> with the update.
>>>>
>>>> Nothing in the server.log files indicates that there is a problem.
>>>>
>>>> Anyone have any ideas what is going on?
>>>>
>>>> Thanks,
>>>> Carla
>>>>
>>>>
>>>> java.lang.Exception: java.net.ConnectException: Connection refused
>>>> at
>>>> test.classloader.hellojsf.helloJSFTestNG.jsfAppDeployedFirstPagetest(helloJSFTestNG.java:122)
>>>>
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>
>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>> at
>>>> org.testng.internal.MethodHelper.invokeMethod(MethodHelper.java:604)
>>>> at org.testng.internal.Invoker.invokeMethod(Invoker.java:470)
>>>> at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:564)
>>>> at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:830)
>>>> at
>>>> org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
>>>>
>>>> at
>>>> org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
>>>> at org.testng.TestRunner.runWorkers(TestRunner.java:678)
>>>> at org.testng.TestRunner.privateRun(TestRunner.java:624)
>>>> at org.testng.TestRunner.run(TestRunner.java:495)
>>>> at org.testng.SuiteRunner.runTest(SuiteRunner.java:300)
>>>> at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:295)
>>>> at org.testng.SuiteRunner.privateRun(SuiteRunner.java:275)
>>>> at org.testng.SuiteRunner.run(SuiteRunner.java:190)
>>>> at org.testng.TestNG.createAndRunSuiteRunners(TestNG.java:792)
>>>> at org.testng.TestNG.runSuitesLocally(TestNG.java:765)
>>>> at org.testng.TestNG.run(TestNG.java:699)
>>>> at org.testng.TestNG.privateMain(TestNG.java:824)
>>>> at org.testng.TestNG.main(TestNG.java:802)
>>>> Caused by: java.net.ConnectException: Connection refused
>>>> at java.net.PlainSocketImpl.socketConnect(Native Method)
>>>> at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
>>>> at
>>>> java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
>>>> at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
>>>> at java.net.Socket.connect(Socket.java:520)
>>>> at java.net.Socket.connect(Socket.java:470)
>>>> at sun.net.NetworkClient.doConnect(NetworkClient.java:157)
>>>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:387)
>>>> at sun.net.www.http.HttpClient.openServer(HttpClient.java:522)
>>>> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:639)
>>>> at
>>>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:957)
>>>>
>>>> at
>>>> java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:367)
>>>> at
>>>> test.classloader.hellojsf.helloJSFTestNG.jsfAppDeployedFirstPagetest(helloJSFTestNG.java:97)
>>>>
>>>> ... 22 more
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>