dev@jsr311.java.net

Re: JSR311: TCK Coverage Document <was> Re: JSR311: Preparing for 1.0 FCS

From: Paul Sandoz <Paul.Sandoz_at_Sun.COM>
Date: Fri, 05 Sep 2008 09:20:19 +0200

Hi,

Not so early as i anticipated but here is the results summary:

   206 tests passed on the following platforms:

   Windows XP SP2,
   RedHat 4.0
   Solaris Sparc 10
   MacOS 10.5

   including signature test, API tests, and End-To-End Tests.

   Tests are run using Java SE 5 and Java SE 6.
   GlassFish V3 build is used as servlet web server.

Paul.

On Aug 29, 2008, at 11:11 PM, Paul Sandoz wrote:

> Hi,
>
> Please see below for the current TCK coverage document. RI results
> will follow early next week.
>
> Paul.
>
>
> TCK Coverage Document for JSR-311 JAX-RS: JavaŠ API for RESTful Web
> Services
>
>
> TCK Components:
> ---------------
> - User's Guide
> - Compatibility Rules
> - Configuration Instructions
> - Test Suite
> - API Tests
> - Signature Tests
> - End-to-End Tests
> - Framework Code
> - JavaTest TM Harness
>
>
> Terminology of Metrics
> ----------------------
> - Assertion: A specific statement of functionality or behavior
> derived from a specification.
> A testable assertion is one that can be validated in an
> implementation by testing.
> - Test: A binary application (or script) comprised of one or more
> Test Cases.
> - Test Case: A single set of test inputs, execution conditions, and
> expected results
> developed to verify an implementation's conformance with a
> specific assertion.
> - Specification Assertion Coverage: Ratio of all assertions tested
> by at least one test
> case to the total number of testable assertions defined by the
> specification.
> - API Coverage: Ratio of methods directly exercised by test cases to
> the total number
> of methods defined by the specification.
>
>
> Coverage
> --------
> - Total testable assertions derived from JAX-RS Specification and
> Javadoc:
>
> 96 specification assertions
> + 296 API assertions
> -------------------------------
> 392 testable assertions
>
> - Total testable assertions tested
>
> 29 specification assertions tested
> + 154 API assertions tested
> --------------------------------------
> 183 testable assertions tested
>
> - 30% of testable specification assertions tested
> - 52% of testable API assertions tested
> - 47% of all (specification and API) assertions tested
>
> - Assertions were identified through the use of CTS Tools and hand
> markup
>
> - Number of tests in JAX-RS TCK: 200 This total may change slightly
> before the TCK ships.
>
> - API Signature Coverage: 100% for all defined public and protected
> members and validated
> by the signature tests included with the JAX-RS TCK.
>
>
> Quality Assurance
> -----------------
> - JAX-RS TCK was run using representative configurations of the
> Reference Implementation on
> the following platforms, either Java 2 SE 5.0 or Java 2 SE 6.0 used:
> * Solaris 10 sparc
> * Red Hat (RHEL) 4.0
> * Windows XP Professional
> * Mac OS 10.5
> - Code quality was demonstrated through the use of code reviews and
> inspections
> - User's Guide was constructed from the standard TCK User's Guide
> template
> - Documentation instructions were verified and tested
>
>
> Justification of Adequacy
> -------------------------
> The JAX-RS TCK version 1 provides a comprehensive set of tests to
> ensure
> all implementations of the JAX-RS specifications are compatible. As
> with all
> TCKs it is impossible to provide tests for 100% assertion coverage.
> Note, while
> the TCK may not have tests for all assertions in the JAX-RS
> specification, all
> implementations must be compatible with the specification.
>
> Untested assertions are due to the effects of late specification
> changes, and the
> availability and cost of test development resources.
>
>
> On Aug 27, 2008, at 10:38 PM, Marc Hadley wrote:
>
>> I just created a new 0.11 snapshot:
>>
>> Spec: https://jsr311.dev.java.net/drafts/spec20080827.pdf
>> Javadoc: https://jsr311.dev.java.net/nonav/releases/0.11/index.html
>> SVN tag: https://jsr311.dev.java.net/source/browse/jsr311/tags/jsr311-api-0.11/
>>
>> The bits are also available in a Maven 2 repo at:
>>
>> http://download.java.net/maven/2/javax/ws/rs/jsr311-api/0.11/
>>
>> This is our candidate 1.0 spec and API and we're now making the
>> final few tweaks to bring the RI and TCK into line. Here's the plan
>> from now:
>>
>> - End of this week or start of next (9/2 latest), send the TCK
>> coverage report and RI results to this list
>> - End of next week or start of following (9/8 latest), submit the
>> spec, api, RI and TCK to the JCP to start the EC final approval
>> ballot. The ballot is open for two weeks.
>> - Week of 9/22 (if we pass the ballot), submit the final materials
>> to the JCP for publication
>> - Week of 10/6 final spec published.
>>
>> We'd hoped to be able to provide access to the TCK before we got to
>> this stage but apparently we are the first JSR to request this and
>> the legal gears grind too slowly to make it possible. EC members
>> will have access to the TCK once the final approval ballot starts.
>> Java EE licensees will have access to the TCK once the spec goes
>> final and we're working on an eval license for non-EE licensees.
>>
>> Marc.
>>
>> ---
>> Marc Hadley <marc.hadley at sun.com>
>> CTO Office, Sun Microsystems.
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_jsr311.dev.java.net
>> For additional commands, e-mail: dev-help_at_jsr311.dev.java.net
>>
>