dev@ws-test-harness.java.net

Re: e2e test harness -- now with better error output!

From: Kohsuke Kawaguchi <Kohsuke.Kawaguchi_at_Sun.COM>
Date: Mon, 23 Oct 2006 21:29:38 -0700

Thanks Ken, and sorry Fabian.

Just one comment about the code. I believe Classes/classes.jar is in the
bootstrap classloader, so I don't think you have to add anything, to
make it work with Mac.

Ken Hofsass wrote:
> Fabian, just to be clear, by "checked in code", I meant that the new and
> hopefully improved jar files have been integrated into the main WSIT e2e
> workspace 'lib' directory... you don't have to build it from the
> test-harness sources or anything. Refreshing your workspace and running
> 'ant clean main' or 'ant clean main-hudson' will then run the e2e tests.
>
> regards,
> Ken
>
> Ken Hofsass wrote:
>
>> Fabian, I've checked in code that I think should handle the OS X
>> variance (code now tests for lib/tools.jar & Classes/classes.jar and
>> adds either if it exists). Please try it out and see if I got the
>> path to classes.jar correct.
>>
>> thanks,
>> Ken
>>
>> Fabian Ritzmann wrote:
>>
>>> Kohsuke Kawaguchi wrote:
>>>
>>>> Fabian Ritzmann wrote:
>>>>
>>>>> Isn't that the perfect excuse to get yourself a Mac. :-)
>>>>
>>>>
>>>>
>>>> Tell that to Vella, please.
>>>
>>>
>>>
>>> My pleasure. :-)
>>>
>>>>> $ find
>>>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>>>>> -follow -name tools.jar
>>>>> $
>>>>>
>>>>> So much about depending on implementation details... :-)
>>>>
>>>>
>>>>
>>>> I know. The good news is in Mustang this is fixed.
>>>
>>>
>>>
>>> The bad news for me is that Mustang will likely require to buy a new
>>> version of Mac OS X. :-(
>>>
>>> What exactly is fixed in Mustang?
>>>
>>>>> There seems to be a classes.jar that contains all com.sun and
>>>>> java/javax classes. You can find it in
>>>>> $JAVA_HOME/../Classes/classes.jar.
>>>>
>>>>
>>>>
>>>> Ca you check if it contain javac? But then you say it contains all
>>>> java/javax classes. Is this the equivalent of rt.jar in
>>>> Unix/Windows? If so, it should be in the bootstrap classloader, so
>>>> all we need to do is to relax the error check a bit.
>>>
>>>
>>>
>>> Not sure what I should be looking for but I definitely see a number
>>> of com.sun.tools.javac and sun.tools.javac classes in there. I can't
>>> easily check what's on the bootstrap classpath but I would expect to
>>> find it there.
>>>
>>>> Does that mean in Mac every JRE comes with javac?
>>>
>>>
>>>
>>> There is only one bundle they call J2SE and it contains a complete JDK.
>>>
>>>>>>> Can't we run all tests and then return with a non-0 exit code if
>>>>>>> anything failed?
>>>>>>
>>>>>>
>>>>>>
>>>>>> That's exactly what we are doing right now. The consequence is
>>>>>> that the build script cannot set failonerror="true", because that
>>>>>> would kill the build as long as any test fail.
>>>>>>
>>>>>> The discussion in this thread was that you don't want that to
>>>>>> happen, but you do want the build to break if there's a
>>>>>> catastrophic errors in running a test, like this one.
>>>>>>
>>>>>> So I suggested that we only return non-0 when there's a
>>>>>> catastrophic failures.
>>>>>
>>>>>
>>>>>
>>>>> Yes, that makes sense. Or could we do it such that the harness
>>>>> returns an error code if there had been a fatal failure or a test
>>>>> failure, then we run the junitreport task and only after that we
>>>>> fail? Something along these lines:
>>>>>
>>>>> <java jar="${test.lib.dir}/harness.jar" fork="true"
>>>>> resultproperty="test.returncode"/>
>>>>> ..
>>>>> <junitreport/>
>>>>> ..
>>>>> <condition property="test.failed">
>>>>> <not>
>>>>> <equals arg1="0" arg2="test.returncode"/>
>>>>> </not>
>>>>> </condition>
>>>>> <fail if="test.failed/>
>>>>
>>>>
>>>>
>>>> That could work, too, but with this approach, a catastrophic failure
>>>> would show up with "no test failure" --- it's just that there was no
>>>> test, but I think it could be confusing.
>>>
>>>
>>>
>>> Yes, that would be confusing. Would it make sense to work with
>>> different error codes for fatal failures and for test failures?
>>>
>>> Fabian
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_ws-test-harness.dev.java.net
>> For additional commands, e-mail: dev-help_at_ws-test-harness.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_ws-test-harness.dev.java.net
> For additional commands, e-mail: dev-help_at_ws-test-harness.dev.java.net
>
>


-- 
Kohsuke Kawaguchi
Sun Microsystems                   kohsuke.kawaguchi_at_sun.com