Yes, your suggestions were discussed in another thread. But when you
look from an overall picture, QL is for developer, RE (using QL hudson
jobs) and dev-build hudson job, QE hudson job(for different bundles of
V3). The requirements for QL should consider all factors and balance the
needs. For example, the hudson jobs will only archive the tests results
and show the trend plot for build process ending with SUCCESSFUL. If the
QL finnal build status is switched to FAILURE when there are test
failures, hudson jobs will not archive test results and show trend plot.
This can cause more confusions. Please see
http://hudson.sfbay/job/v3-quicklook/9625/ for example.
The enhance in my mind is to move the summary of testng results close to
the bottom so that it will be more obvious. As for issue/RFE filing
against QL, it's good for tracking purposes and prioritizing the
requests. I'll take the original ownership of the issue/RFE of QL. QL is
in v3 open source project and people with interests are encourged to
make contributions to it. In fact if you look at the commit logs of QL,
you'll see a long list of people made contributions for tests or
framework and increased the coverage of QL recently (Thanks!). The docs
at quicklook/QuickLook_Test_Instructions.html is getting updated
regularly and users should be able to go from there to use/add tests to
QL. You are very welcome to make contributions to QL and/or take
ownership of QL RFEs.
Thanks,
Ming
Sahoo wrote:
>
>
> Ming Zhang wrote:
>> Please refer to the mail I reply earlier to Tim. Here are more details.
>>
>> The QL basically does the following:
>>
>> 1. Start up Derby.
>> 2. Start up V3.
>> 3. Build test APPs.
>> 4. Create Users.
>> 5. Deploy tests.
>> 6. Compile test client.
>> 7. Run tests and generate test report in quicklook/test-output-web
>> 8. Undeploy test APPs.
>> 9. Delete users.
>> 10. Stop V3.
>> 11. Stop Derby.
>>
>> In Tim's case, if the QL build/testing process was forced to fail in
>> the middle of testing, the test results will be partial. I don't
>> think the QL tests should stop for tests failing in AMX alone. Also,
>> V3 won't be clean in that case. It would have testing APPs still
>> deployed, testing users left over, V3 and Derby process running.
>> These are not ideal testing scenarios for implementing automated QL
>> Hudson job.
> The final status should indicate FAILURE along with a suitable status
> code indicating the cause/nature of failure if any of the above steps
> fail. We have already discussed this as part of basic requirement of a
> test framework, haven't we?
>>
>> Please file issue/RFE in issue tracker if you still think this is a bug.
> What's the point? Will you fix it? If yes, why do you care about a bug
> being filed? Is this discussion not enough?
>
> Thanks,
> Sahoo
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>