dev@ws-test-harness.java.net

Re: e2e test harness -- now with better error output!

From: Kohsuke Kawaguchi <Kohsuke.Kawaguchi_at_Sun.COM>
Date: Mon, 16 Oct 2006 08:49:47 -0700

Fabian Ritzmann wrote:
> Kohsuke Kawaguchi wrote:
>> Ken Hofsass wrote:
>>> Fabian,
>>>
>>> Fabian Ritzmann wrote:
>>>
>>>> [java] Caused by: java.io.IOException: No such file:
>>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/lib/tools.jar
>>>>
>>>> [java] at com.sun.xml.ws.test.Realm.addJar(Realm.java:68)
>>>> [java] at com.sun.xml.ws.test.Main.fillWorld(Main.java:305
>>>
>>>> The path in the java.io.IOException is indeed wrong and does not
>>>> correspond to what my JAVA_HOME env variable is set to.
>>>
>>> All the harness is doing to derive that path is to call
>>> System.getProperty(java.home) and append 'lib/tools.jar' and that
>>> section of code hasn't changed with recent file changes. Any idea
>>> where it could be finding that value?
>>
>> Looks like on Mac the layout is different. Fabian should be able to
>> tell us where we can load tools.jar on Mac.
>
> The JAVA_HOME is
> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
> or better
> /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home
>
> Ant says:
> Detected Java version: 1.5 in:
> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>
> It used to work with the previous test harness. No idea why it's
> suddenly picking up the wrong home.

Our code uses "java.home" system property and looks for ../lib/tools.jar
. This works on Windows and Unix systems, where java.home points to JRE
in JDK. But this must be different in Mac.

None of us have Macs except you, so we need your help to get more info.

1. Please tell us what's your:
     System.out.println(System.getProperty("java.home"));

2. Please tell us where's your tools.jar, relative to java.home.

>>>> One other thing is that the Ant build should not report a successful
>>>> build in that case. You should probably set failonerror="true" when
>>>> invoking java in the run-tests macro.
>>>
>>> I agree the fact that it says "successful" is misleading. However, we
>>> don't generally want the test run to quit when one test fails/errors.
>>> Rather, we want to get the full set of results. Maybe there's a way
>>> to tell ant just to keep its fat mouth shut about 'success'. Or am I
>>> missing your point?
>>
>> Perhaps we should return exit code 0 even if some tests fail, and use
>> non-0 exit code only for fatal problems.
>
> Can't we run all tests and then return with a non-0 exit code if
> anything failed?

That's exactly what we are doing right now. The consequence is that the
build script cannot set failonerror="true", because that would kill the
build as long as any test fail.

The discussion in this thread was that you don't want that to happen,
but you do want the build to break if there's a catastrophic errors in
running a test, like this one.

So I suggested that we only return non-0 when there's a catastrophic
failures.

-- 
Kohsuke Kawaguchi
Sun Microsystems                   kohsuke.kawaguchi_at_sun.com