Hi,
On Tue, Nov 4, 2014 at 5:48 PM, Paul Hendley <paul.hendley_at_oracle.com> wrote:
> The (JASPIC) TCK specifies it's own AuthConfigFactory. When the appserver
> starts, our AuthConfigFactory is loaded. It then reads an XML file and uses
> the info from that xml file to register our auth config providers. The
> server auth modules (SAMs) are not loaded from XML files but instead are
> loaded directly when our tcks providers get loaded. So we load ACP's from
> an xml file but not SAMs.
Thanks a lot for giving that insight Paul. It's really interesting to
have some idea of how the TCK approaches testing.
> The largest focus of the JASPIC TCK was to test the runtimes ability to
> interact with AuthConfigFactorys and AuthConfigProviders's as well as some
> of the interactions with auth context and auth configs. We chose not to do
> much with the SAMs instead focusing on the other components because we
> didn't see many ways to validate the testable assertions in each vendors
> SAM.
I hear you, but as it appears there were quite some things that could
be tested ;)
Maybe one of the most important things is that the caller principal
callback is actually handled by the container provided handler. I
think this is the one aspect of JASPIC that's absolutely key to its
functioning. Now one particular certified server didn't implement this
(CallbackHandler#handle was simply a noop). As a result, no actual SAM
could ever work on it, despite that server being fully certified.
> If folks believe the TCK is missing any coverage or there are areas
> you are unsure of, please pass the info along.
I captured an initial batch of such areas in a series of tests that I
created here:
https://github.com/javaee-samples/javaee7-samples/tree/master/jaspic
These tests test for things that actually differed between the various
JASPIC implementations that I investigated (GlassFish, WebLogic, JEUS,
WebSphere, Geronimo and JBoss).
Specifically they check for things like whether the SAM is actually
invoked for both public and protected resources (some servers only did
this for protected ones) and whether a SAM is invoked again after
authentication had been done before (some servers automatically
remembered authentication), and a very remarkable one that needed a
test; whether secureResponse is called after the resource invocation
(for some reason quite a number of servers called it before the
resource invocation, right after validateRequest), whether a request
and response wrapped by a SAM actually arrives at the resource/Servlet
(surprisingly this initially worked with none of the servers that I
tested), etc.
All those things (and few more) were most likely not tested in the
TCK, since for each of them there was at least one certified server
that failed a test for it.
> Arjan, I know you and Ron Monzillo had some discussions on possible tests to
> add to the JASPIC TCK. We did add a couple new JASPIC tests based on your
> input.
That's really great to hear :) Indeed, I discussed quite a number of
the cases that my tests test with Ron Monzillo, both to verify that my
understanding of JASPIC was correct and the tests where indeed testing
the right thing, and for having additional tests included in the TCK.
In as far as they haven't been added yet, I would propose to add tests
for everything that my tests are now testing. It's still a rather
basic test set and doesn't include everything yet (e.g. I still
haven't added tests for SAMs doing forwards and includes), but I think
it's a good addition.
Kind regards,
Arjan Tijms