Lloyd L Chambers wrote:
> Kohsuke,
>
> Sounds promising, though I don't fully understand the proposal.
>
> I see 3 levels of testing:
> - true unit tests that can build and run as part of the normal build
> - end-to-end tests closely tied to a particular area of the server (my
> AMX tests)
> - full-blown "the world" tests that setup clusters etc.
>
> There might be crossover between the latter two.
>
> In the short term my problem is building the AMX tests...I very much
> like the automation provided by the src/tests directory, so the
> question is whether that same ease of use can be applied to my end to
> end tests. This is where my understanding breaks down of your
> response below.
My proposal is to place this in its own Maven module outside the main
GFv3 build tree, so it won't go into your src/tests directory.
But with a little bit of Maven mojo development, I think a similar ease
of use can be attained. Something like:
- you run "mvn test" on your test module to execute it against bits
in your local Maven repository.
- you run "mvn test -Dhome=path/to/gfv3" to run tests on the existing
GFv3 installation somewhere on the disk.
- you run "mvn install" and it pushes the stand-alone runnable test jar
to the maven repository
- the stand-alone runnable test jar can be invoked like
"java -jar amx-test.jar path/to/gfv3" to run GFv3.
> Thanks,
> Lloyd
>
> On Apr 15, 2008, at 2:12 PM, Kohsuke Kawaguchi wrote:
>>
>> I think tests like that better live in its own directory with some
>> kind of build script --- essentially the same set up as quick look
>> test.
>>
>> Since this is an end-to-end test, I think it's actually more
>> valuable that you can take any v3 bundle built somewhere and run the
>> tests.
>>
>> For example, when we modify HK2, Hudson can build a custom GFv3 +
>> this bleedging edge HK2, and then it can run your tests, before the
>> rest of GFv3 even sees this new build of HK2. In this way, by the
>> time HK2 gets integrated, chances of that breaking AMX will be much
>> much less.
>>
>> Also, if the test itself is pushed to the Maven repository, then we
>> might be able to hook up execution of tests in several other places,
>> such as during the normal build of AMX.
>>
>>
>> I think there's a value in having a good harness that runs end-to-
>> end test, building on top of JUnit. Its usefulness goes far beyond
>> just AMX.
>>
>>
>> Lloyd L Chambers wrote:
>>> I am currently trying to figure out how to port V2 AMX tests to V3.
>>> In V3, how can we support tests other than plain vanilla unit tests?
>>> - might or might not be true unit tests
>>> - might or might not be junit tests
>>> - might need special setup to execute.
>>> What I would like is formal support for compiling (and optionally
>>> executing) test code that lives other than under the test/ directory.
>>> More details--
>>> In V3 we have a structure that supports unit tests--source code
>>> goes under src/main/java and test code goes under src/test/java.
>>> The build compiles both automagically--very nice!
>>> In V2 I developed a fairly large set of tests for the AMX
>>> management API. While the tests are not unit tests in the usual
>>> sense, they junit.framework.TestCase tests, and run (in V2) using
>>> the junit test framework. The tests do *end-to-end* testing from
>>> a client to a running server, and given the inherent nature of
>>> what needs to be tested, this is the best approach for this
>>> particular case; mock objects offer little value since the goal is
>>> to test the end-to-end behavior, not some isolated piece of code.
>>> So let me call these "End to End Tests" or EETs. These don't
>>> work when placed in the test/ directory, since they require a
>>> test environment with a full running server--they'll all just
>>> fail. So I can't put my test code under tests/. Annotating them
>>> with @Ignore is no good either, because they *must* be run when
>>> the right setup is in place.
>>> Unfortunately, if the test code does not live under test/, then it
>>> has to live elsewhere, along with some kind of build script. That
>>> means making that script rely on build products of the regular
>>> build, eg duplicating and maintaining knowledge found elsewhere in
>>> the build.
>>> ---
>>> Lloyd L Chambers
>>> lloyd.chambers_at_sun.com
>>> Sun Microsystems, Inc
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>>
>> --
>> Kohsuke Kawaguchi
>> Sun Microsystems kohsuke.kawaguchi_at_sun.com
>
> ---
> Lloyd L Chambers
> lloyd.chambers_at_sun.com
> Sun Microsystems, Inc
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>
>
--
Kohsuke Kawaguchi
Sun Microsystems kohsuke.kawaguchi_at_sun.com