dev@ws-test-harness.java.net

Re: e2e test harness -- now with better error output!

From: Ken Hofsass <Kenneth.Hofsass_at_Sun.COM>
Date: Mon, 23 Oct 2006 16:41:31 -0700

Fabian, just to be clear, by "checked in code", I meant that the new and
hopefully improved jar files have been integrated into the main WSIT e2e
workspace 'lib' directory... you don't have to build it from the
test-harness sources or anything. Refreshing your workspace and running
'ant clean main' or 'ant clean main-hudson' will then run the e2e tests.

regards,
Ken

Ken Hofsass wrote:

> Fabian, I've checked in code that I think should handle the OS X
> variance (code now tests for lib/tools.jar & Classes/classes.jar and
> adds either if it exists). Please try it out and see if I got the
> path to classes.jar correct.
>
> thanks,
> Ken
>
> Fabian Ritzmann wrote:
>
>> Kohsuke Kawaguchi wrote:
>>
>>> Fabian Ritzmann wrote:
>>>
>>>> Isn't that the perfect excuse to get yourself a Mac. :-)
>>>
>>>
>>>
>>> Tell that to Vella, please.
>>
>>
>>
>> My pleasure. :-)
>>
>>>> $ find
>>>> /System/Library/Frameworks/JavaVM.framework/Versions/1.5.0/Home
>>>> -follow -name tools.jar
>>>> $
>>>>
>>>> So much about depending on implementation details... :-)
>>>
>>>
>>>
>>> I know. The good news is in Mustang this is fixed.
>>
>>
>>
>> The bad news for me is that Mustang will likely require to buy a new
>> version of Mac OS X. :-(
>>
>> What exactly is fixed in Mustang?
>>
>>>> There seems to be a classes.jar that contains all com.sun and
>>>> java/javax classes. You can find it in
>>>> $JAVA_HOME/../Classes/classes.jar.
>>>
>>>
>>>
>>> Ca you check if it contain javac? But then you say it contains all
>>> java/javax classes. Is this the equivalent of rt.jar in
>>> Unix/Windows? If so, it should be in the bootstrap classloader, so
>>> all we need to do is to relax the error check a bit.
>>
>>
>>
>> Not sure what I should be looking for but I definitely see a number
>> of com.sun.tools.javac and sun.tools.javac classes in there. I can't
>> easily check what's on the bootstrap classpath but I would expect to
>> find it there.
>>
>>> Does that mean in Mac every JRE comes with javac?
>>
>>
>>
>> There is only one bundle they call J2SE and it contains a complete JDK.
>>
>>>>>> Can't we run all tests and then return with a non-0 exit code if
>>>>>> anything failed?
>>>>>
>>>>>
>>>>>
>>>>> That's exactly what we are doing right now. The consequence is
>>>>> that the build script cannot set failonerror="true", because that
>>>>> would kill the build as long as any test fail.
>>>>>
>>>>> The discussion in this thread was that you don't want that to
>>>>> happen, but you do want the build to break if there's a
>>>>> catastrophic errors in running a test, like this one.
>>>>>
>>>>> So I suggested that we only return non-0 when there's a
>>>>> catastrophic failures.
>>>>
>>>>
>>>>
>>>> Yes, that makes sense. Or could we do it such that the harness
>>>> returns an error code if there had been a fatal failure or a test
>>>> failure, then we run the junitreport task and only after that we
>>>> fail? Something along these lines:
>>>>
>>>> <java jar="${test.lib.dir}/harness.jar" fork="true"
>>>> resultproperty="test.returncode"/>
>>>> ..
>>>> <junitreport/>
>>>> ..
>>>> <condition property="test.failed">
>>>> <not>
>>>> <equals arg1="0" arg2="test.returncode"/>
>>>> </not>
>>>> </condition>
>>>> <fail if="test.failed/>
>>>
>>>
>>>
>>> That could work, too, but with this approach, a catastrophic failure
>>> would show up with "no test failure" --- it's just that there was no
>>> test, but I think it could be confusing.
>>
>>
>>
>> Yes, that would be confusing. Would it make sense to work with
>> different error codes for fatal failures and for test failures?
>>
>> Fabian
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_ws-test-harness.dev.java.net
> For additional commands, e-mail: dev-help_at_ws-test-harness.dev.java.net
>