dev@ws-test-harness.java.net

Re: e2e test harness -- now with better error output!

From: Ken Hofsass <Kenneth.Hofsass_at_Sun.COM>
Date: Mon, 23 Oct 2006 16:36:46 -0700

Fabian, I've checked in code that I think should handle the OS X
variance (code now tests for lib/tools.jar & Classes/classes.jar and
adds either if it exists). Please try it out and see if I got the path
to classes.jar correct.

thanks,
Ken

Fabian Ritzmann wrote:

> Kohsuke Kawaguchi wrote:
>
>> Fabian Ritzmann wrote:
>>
>>> Isn't that the perfect excuse to get yourself a Mac. :-)
>>
>>
>> Tell that to Vella, please.
>
>
> My pleasure. :-)
>
>>> $ find
>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>>> -follow -name tools.jar
>>> $
>>>
>>> So much about depending on implementation details... :-)
>>
>>
>> I know. The good news is in Mustang this is fixed.
>
>
> The bad news for me is that Mustang will likely require to buy a new
> version of Mac OS X. :-(
>
> What exactly is fixed in Mustang?
>
>>> There seems to be a classes.jar that contains all com.sun and
>>> java/javax classes. You can find it in
>>> $JAVA_HOME/../Classes/classes.jar.
>>
>>
>> Ca you check if it contain javac? But then you say it contains all
>> java/javax classes. Is this the equivalent of rt.jar in Unix/Windows?
>> If so, it should be in the bootstrap classloader, so all we need to
>> do is to relax the error check a bit.
>
>
> Not sure what I should be looking for but I definitely see a number of
> com.sun.tools.javac and sun.tools.javac classes in there. I can't
> easily check what's on the bootstrap classpath but I would expect to
> find it there.
>
>> Does that mean in Mac every JRE comes with javac?
>
>
> There is only one bundle they call J2SE and it contains a complete JDK.
>
>>>>> Can't we run all tests and then return with a non-0 exit code if
>>>>> anything failed?
>>>>
>>>>
>>>> That's exactly what we are doing right now. The consequence is that
>>>> the build script cannot set failonerror="true", because that would
>>>> kill the build as long as any test fail.
>>>>
>>>> The discussion in this thread was that you don't want that to
>>>> happen, but you do want the build to break if there's a
>>>> catastrophic errors in running a test, like this one.
>>>>
>>>> So I suggested that we only return non-0 when there's a
>>>> catastrophic failures.
>>>
>>>
>>> Yes, that makes sense. Or could we do it such that the harness
>>> returns an error code if there had been a fatal failure or a test
>>> failure, then we run the junitreport task and only after that we
>>> fail? Something along these lines:
>>>
>>> <java jar="${test.lib.dir}/harness.jar" fork="true"
>>> resultproperty="test.returncode"/>
>>> ..
>>> <junitreport/>
>>> ..
>>> <condition property="test.failed">
>>> <not>
>>> <equals arg1="0" arg2="test.returncode"/>
>>> </not>
>>> </condition>
>>> <fail if="test.failed/>
>>
>>
>> That could work, too, but with this approach, a catastrophic failure
>> would show up with "no test failure" --- it's just that there was no
>> test, but I think it could be confusing.
>
>
> Yes, that would be confusing. Would it make sense to work with
> different error codes for fatal failures and for test failures?
>
> Fabian
>