dev@ws-test-harness.java.net

Re: e2e test harness -- now with better error output!

From: Fabian Ritzmann <Fabian.Ritzmann_at_Sun.COM>
Date: Fri, 20 Oct 2006 03:37:38 +0300

Hi,

Is anybody looking into the first part of my email? I can't really write
an e2e test if I can't execute it on my machine...

Fabian



Fabian Ritzmann wrote:
> Kohsuke Kawaguchi wrote:
>> Fabian Ritzmann wrote:
>>> The JAVA_HOME is
>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>>> or better
>>> /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
>>>
>>> Ant says:
>>> Detected Java version: 1.5 in:
>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>>>
>>> It used to work with the previous test harness. No idea why it's
>>> suddenly picking up the wrong home.
>>
>> Our code uses "java.home" system property and looks for
>> ../lib/tools.jar . This works on Windows and Unix systems, where
>> java.home points to JRE in JDK. But this must be different in Mac.
>>
>> None of us have Macs except you, so we need your help to get more info.
>
> Isn't that the perfect excuse to get yourself a Mac. :-)
>
>> 1. Please tell us what's your:
>> System.out.println(System.getProperty("java.home"));
>
> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>
>> 2. Please tell us where's your tools.jar, relative to java.home.
>
> $ find /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
> -follow -name tools.jar
> $
>
> So much about depending on implementation details... :-)
>
> There seems to be a classes.jar that contains all com.sun and
> java/javax classes. You can find it in $JAVA_HOME/../Classes/classes.jar.
>
>>> Can't we run all tests and then return with a non-0 exit code if
>>> anything failed?
>>
>> That's exactly what we are doing right now. The consequence is that
>> the build script cannot set failonerror="true", because that would
>> kill the build as long as any test fail.
>>
>> The discussion in this thread was that you don't want that to happen,
>> but you do want the build to break if there's a catastrophic errors
>> in running a test, like this one.
>>
>> So I suggested that we only return non-0 when there's a catastrophic
>> failures.
>
> Yes, that makes sense. Or could we do it such that the harness returns
> an error code if there had been a fatal failure or a test failure,
> then we run the junitreport task and only after that we fail?
> Something along these lines:
>
> <java jar="${test.lib.dir}/harness.jar" fork="true"
> resultproperty="test.returncode"/>
> ...
> <junitreport/>
> ...
> <condition property="test.failed">
> <not>
> <equals arg1="0" arg2="test.returncode"/>
> </not>
> </condition>
> <fail if="test.failed/>
>
> Fabian
>