dev@glassfish.java.net

Re: [v3] QL reports success at the end even with failures

From: Sahoo <sahoo_at_sun.com>
Date: Wed, 24 Sep 2008 13:13:35 +0530

Ming Zhang wrote:
> Please refer to the mail I reply earlier to Tim. Here are more details.
>
> The QL basically does the following:
>
> 1. Start up Derby.
> 2. Start up V3.
> 3. Build test APPs.
> 4. Create Users.
> 5. Deploy tests.
> 6. Compile test client.
> 7. Run tests and generate test report in quicklook/test-output-web
> 8. Undeploy test APPs.
> 9. Delete users.
> 10. Stop V3.
> 11. Stop Derby.
>
> In Tim's case, if the QL build/testing process was forced to fail in
> the middle of testing, the test results will be partial. I don't think
> the QL tests should stop for tests failing in AMX alone. Also, V3
> won't be clean in that case. It would have testing APPs still
> deployed, testing users left over, V3 and Derby process running. These
> are not ideal testing scenarios for implementing automated QL Hudson job.
The final status should indicate FAILURE along with a suitable status
code indicating the cause/nature of failure if any of the above steps
fail. We have already discussed this as part of basic requirement of a
test framework, haven't we?
>
> Please file issue/RFE in issue tracker if you still think this is a bug.
What's the point? Will you fix it? If yes, why do you care about a bug
being filed? Is this discussion not enough?

Thanks,
Sahoo