Hi Priti,
Please find my answers inline.
Thanks,
Swarna Mishra
>From: Priti Tiwary <Priti.Tiwary_at_Sun.COM>
>Reply-To: admin_at_glassfish.dev.java.net
>To: SwrnaLata Mishra <swrna_mishra_at_hotmail.com>
>CC: Ken.Paulsen_at_Sun.COM, admin_at_glassfish.dev.java.net
>Subject: Re: Regarding the Framework
>Date: Thu, 05 Apr 2007 14:01:21 -0700
>Hi Swrna,
>Thanks a lot for clarifying what are the exact topics your "test framework
>development" will cover.
>
>1) In the present setup, i too had realised that sometimes testcases and
>hence methods related to that test case becomes quite big and it is best if
>we have a different file for each test. It does make the test framework
>scalable.
>But what will be the criteria to group tests? We can have installGroup,
>UninstallGroup, TreeGroup, OperateGroup. Have you chalked that out?
On a macro level test groups can be Nightly Tests (integrated with nightly
builds), regression tests (to run a system test cycle) etc. Within a group,
we can arrange tests into class files. Recently Andre has sent some test
files like TestWarDeploy.java, TestEarDeploy.java etc, and he has nicely
arranged the test cases. I like his approach, and I propose that we should
follow this format.
>Then installGroup can have happy path use cases, invalid/negative use cases
>using all sort of archives. Anissa had once sent us a list of valid
>archives which can be used for testing. If we check that in and use in our
>test cases. So each test group should have tests for happy paths and
>unhappy paths.
>Having test group specific packages will make a test developer understand
>where to place his file more clearly.
>Right now the package for test files is
>com.sun.jbi.jsf.test
>I reused the package name from previous HtmlUnit/Rhino Unit JBI tests which
>we had before.
>We will need to decide on a better package structure also based on
>"testGroups"
Packaging will help us identify the test cases easily. I agree with your
observation.
We can continue with com.sun.jbi.jsf.test, however the .jsf is a bit
irrelevent here. May be we can have com.sun.jbi.selenium.test.admingui
package. Please comment and we will start using the common package.
>Yes i initially had tried separating tests into different files and faced
>issues. The trickier part will be to tell Selenium that after exiting the
>previous test, which frame/window to choose to start the next test. Similar
>issues i addressed when i had several tests in one file and had to switch
>from one test method to other .
>
We will have to keep 'logically together' test cases in a single file. For
example test cases related to deploying a .war file can be in a single
class. At the end of these test cases, we wil log out from the AdminGUI.
When the next class execution starts (for example deploy .rar file related
cases), we will login again and will navigate to the required frame/window.
>2) Having a parameter file is a good idea. We can use option to ant target
>and read that too. I did see that several people suggested "store" method
>to store the hardcoded values in variables and reuse it. But i did not find
>it was supported in SeleniumIDE tests which i developed before. This will
>also avoid any hardcoded path references in test.
>
I think we are on the same page here. My current implementation of framework
already support the property file. It is very similar to what Andre has
provided in his TestConfiguration.java
>3) Regarding logging, certainly we need to redirect test output to a file .
>Right now i used to follow the exception's getMessage() method to see the
>error. We might have to add logic to filter out TimedOut exceptions/errors
>which are pretty common and should not worry a user. The timedout
>exceptions could cause a preceding test to fail sometimes when it becomes a
>reason to worry. Only errors like Element not found, should worry the users
>
I will take care of these points in my framework.
>I would like to add one more thing in our test development TODO list.
>Robustness in repeatability.
>This is an issue i feel needs attention, whenever a user develops a
>testcase. My Solaris machine runs tests slow and needs more time for page
>to load (selenium.waitForPageToLoad(1000)) and timesout and cannot find
>elements on a page, even when it is there, while it works fine for windows
>. Same test or timedout parameter works fine for one run but not for other
>run in same setup. So the tests should provide a reproducible path.
>
This is a perfect example of what we need to define in our property file. If
this is made configurable in the input property file, then we can modify the
waitForPageToLoad for different platforms without recompiling the code.
>Targetting firefox using chrome url is my highest priority as it can be run
>on Solaris as well as Windows. And i suppose ADMINBUI is a typo, as i even
>saw that in the tests you developed. It is actually Admin-GUI..where GUI
>stands for Graphical user interface.
>
:) BUI refers to Browser User Interface, but its a trivial point to discuss.
from hereon I will also use GUI to maintain consistency.
>Documentation is certainly important part of the whole project but can be
>addressed later. I think for the time being, if we can address
When I referred to documentation, I meant by maintaining a link between our
defined test cases (typically in a test case doc) and the test cases
implemented in our .java files. This is referred as traceability matrix in
the system testing world. This becomes useful when the requirements change
and we have to update our test suite accordingly. Thus if we are modifying a
test case definition in the test case doc, we should come to know what .java
files will be impacted. May be we need to research and start using a Test
management tool. I will take care of this.
_________________________________________________________________
Mortgage rates near historic lows. Refinance $200,000 loan for as low as
$771/month*
https://www2.nextag.com/goto.jsp?product=100000035&url=%2fst.jsp&tm=y&search=mortgage_text_links_88_h27f8&disc=y&vers=689&s=4056&p=5117