Ken Hofsass wrote:
> Hi Kohsuke,
>
> Thanks for your suggestions. That sounds fine to me. I'll work on making
> that change. I was looking at the version-checking code... the harness
> already accepts 'applicable versions' for the test and for the indvidual
> clients. But no comparisons are ever done. That's easy to add.
I think this is a good addition anyway. For example already in the
JAX-WS RI, we need to maintain 2.0 lines and 2.1 lines, and some tests
only apply to 2.1.
> I'm not clear where to pull the version info for the code being tested
> (jaxws/wsit). How do you derive that with JAXB? Is it passed in on
> test command line or from a file? Something in META-INF?
In JAXB, I simply use the command line option (and when none is given, I
run all the tests.) I never really run tests locally, so having an
option wasn't a big problem. We can try to infer that from manifest
entry, but I'm not sure if you can always get it reliably from there. So
I see your point that it's going to be added inconvenience for developers.
I also realized that perhaps the cheapest way to do this is just to
provide an option to run all the skipped tests, and have Hudson run with
it so that it will run all the tests anyway. If the goal is to allow
people to run tests locally and have them all pass, then that might be
enough, and it nicely takes advantage of your investment in the skip
feature.
<rant>
Personally, I don't really believe in leaving broken tests broken. A
part of the reason for having tests it to appeal to the human nature of
trying to keep things clean, and hence it's essential to see failures as
failures (I've heard that there's one weight reduction methodology where
you' just have people measure their weight everyday --- I think it works
for a similar reason.)
Letting people marking a test as skipped is like letting people to sweep
the problem under the rug. There's a broken window effect, too.
</rant>
Anyway, if WSIT team wants that skip feature, then I don't want you to
do extra work just because I don't like it.
--
Kohsuke Kawaguchi
Sun Microsystems kohsuke.kawaguchi_at_sun.com