quality@glassfish.java.net

Re: What did not go right...

From: Aditya Dada <Aditya.Dada_at_Sun.COM>
Date: Fri, 01 Dec 2006 18:58:55 -0500

I can give some background of where the gap came in...

When we were writing the framework for Quick Look, we specified the
options on purpose, since not everyone was expected to run the tests
with the defaults.

In fact, even now, when the test will be added, some fancy if-else in
ANT (which doesn't lend itself readily to scripting) would have to be
put in place to make sure that if a non-default port is specified in
config.properties, then --port option IS passed to the deploy command
otherwise the test will fail for some developers out there using
non-default ports.

Also, since SQE tests, smoke tests and Quick Look test frameworks, all
borrow heavily from each other, (and CTS tests don't cover CLI), the gap
we see now, came into the picture.

But this is also a good wakeup call....its time to revisit the
framework, and find some of the common cases that aren't being covered.

-Aditya




vince kraemer changed the world a bit at a time, and said on 12/1/2006
5:06 PM:
> That is really good news.
>
> But the question still stands... What did not go right? Is there a
> gap/blind-spot/root-cause/something that prevented this test from
> being in the test suite that is used to validate a build as "promotable"?
>
> If there was "something"; will the addition of this test, "use
> asadmin deploy without the port option specified" going to address
> that "something" completely, or do we need to add some more test to
> "close the gap"?
>
> I was very surprised to see this issue in the promoted build,
> partially because I know the number of folks testing the application
> server produced from the Project GlassFish code-base and the HUGE
> number of automated test that the code gets subjected to (on a near
> continuous basis)....
>
> vbk
>
> Judy Tang wrote:
>> I talked with SQE and developer and we plan to add a QL test to test
>> asadmin deploy command
>> without giving --port option.
>>
>> Thanks,
>> Judy
>>
>> vince kraemer wrote:
>>
>>> Over on the dev alias there are reports of an issue that seems like
>>> it would have prevented the build from being promoted...
>>>
>>> https://glassfish.dev.java.net/servlets/ReadMsg?list=dev&msgNo=2792
>>>
>>> The issue did not stop the build from being promoted.
>>>
>>> How can we improve the promotion process to prevent an issue like
>>> this from being present in the promoted build in the future?
>>>
>>> Is it worth doing?
>>>
>>> thanks,
>>> vbk
>>

-- 
"A little risk management saves a lot of fan cleaning."