dev@glassfish.java.net

Re: 7 failures on the latest QL run

From: Marina Vatkina <Marina.Vatkina_at_Sun.COM>
Date: Tue, 16 Sep 2008 15:31:57 -0700

I got it again. With 2.0.7. I was running 'mvn clean; svn update; mvn -U install'.

-marina

Sahoo wrote:
> Jerome Dochez wrote:
>
>> Sahoo wrote:
>>
>>> Are you able to reproduce with mvn 2.0.7?
>>
>> yes unfortunely.
>
> I think we are fortunate. Are you running both clean and install as part
> of same maven reactor? Have you tried running clean and install separately?
>
> Thanks,
> Sahoo
>
>>>
>>> Jerome Dochez wrote:
>>>
>>>> yes this is the usual bug, as identified by Marina
>>>>
>>>> jerome
>>>>
>>>> Sahoo wrote:
>>>>
>>>>> Are you able to reproduce? Does javax.security.jacc.jar contain
>>>>> javax.servlet classes?
>>>>>
>>>>> Thanks,
>>>>> Sahoo
>>>>>
>>>>> Jerome Dochez wrote:
>>>>>
>>>>>> well I just reproduced the problem with a good old 2.0.7....
>>>>>>
>>>>>> Sahoo wrote:
>>>>>>
>>>>>>> A few days we started a thread to this effect. Although we have
>>>>>>> not concluded the discussion, 2.0.7 was mostly favored followed
>>>>>>> by 2.0.9. I personally use 2.0.7.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Sahoo
>>>>>>>
>>>>>>> Marina Vatkina wrote:
>>>>>>>
>>>>>>>> Sahoo,
>>>>>>>>
>>>>>>>> The problem doesn't exist with 2.0.7.
>>>>>>>>
>>>>>>>> BTW, I tried to find the recommended maven version in our build
>>>>>>>> instruction, and it's mentioned only on 2 adminGUI related wiki
>>>>>>>> pages, and on V3FullBuildInstructions, not on
>>>>>>>> V3DevelopmentInstructions or any other pages :(.
>>>>>>>>
>>>>>>>> thanks,
>>>>>>>> -marina
>>>>>>>>
>>>>>>>> Sahoo wrote:
>>>>>>>>
>>>>>>>>> Marina,
>>>>>>>>>
>>>>>>>>> I see you are using mvn 2.0.8, which is known to cause problems
>>>>>>>>> in our build environment. Use mvn 2.0.7. People have said they
>>>>>>>>> are able to use 2.0.9, so you may be able to use 2.0.9 as well.
>>>>>>>>> You can also take it up with maven forum, they are very good in
>>>>>>>>> responding to such issues.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Sahoo
>>>>>>>>>
>>>>>>>>> Marina Vatkina wrote:
>>>>>>>>>
>>>>>>>>>> Yes, 'mvn clean install' if run in v3 directory, results in
>>>>>>>>>> these classes being added to the
>>>>>>>>>> javax.security.jacc-10.0-SNAPSHOT.jar:
>>>>>>>>>>
>>>>>>>>>> javax/servlet/http/HttpServletRequest.class
>>>>>>>>>> javax/servlet/http/Cookie.class
>>>>>>>>>> javax/servlet/http/HttpSession.class
>>>>>>>>>> javax/servlet/http/HttpSessionContext.class
>>>>>>>>>> javax/servlet/ServletRequest.class
>>>>>>>>>> javax/servlet/ServletInputStream.class
>>>>>>>>>> javax/servlet/RequestDispatcher.class
>>>>>>>>>> javax/servlet/ServletContext.class
>>>>>>>>>> javax/servlet/ServletResponse.class
>>>>>>>>>> javax/servlet/ServletException.class
>>>>>>>>>> javax/servlet/Servlet.class
>>>>>>>>>> javax/servlet/ServletOutputStream.class
>>>>>>>>>> javax/servlet/ServletConfig.class
>>>>>>>>>>
>>>>>>>>>> jar created by 'mvn -X -f
>>>>>>>>>> security/javax.security.jacc/pom.xml' didn't include any of
>>>>>>>>>> those classes.
>>>>>>>>>>
>>>>>>>>>> No, I didn't change mvn version since we switched to maven 2.
>>>>>>>>>>
>>>>>>>>>> I'll send you the pointer to the output of 'mvn -X' separately.
>>>>>>>>>>
>>>>>>>>>> thanks,
>>>>>>>>>> -marina
>>>>>>>>>>
>>>>>>>>>> Sahoo wrote:
>>>>>>>>>>
>>>>>>>>>>> Are you able to consistently produce a
>>>>>>>>>>> javax.security.jacc.jar containing javax.servlet classes? Can
>>>>>>>>>>> you try with mvn 2.0.7 to see if it solves the issue? If you
>>>>>>>>>>> have not switched maven version recently, we need to find out
>>>>>>>>>>> what change triggered the build issue.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Sahoo
>>>>>>>>>>>
>>>>>>>>>>> Marina Vatkina wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Sahoo,
>>>>>>>>>>>>
>>>>>>>>>>>> Can it be the wrong (buggy) mvn version? I never changed
>>>>>>>>>>>> from 2.0.8.
>>>>>>>>>>>>
>>>>>>>>>>>> I can rerun the build, or I can try to switch mvn first.
>>>>>>>>>>>> Which version should I use in that case?
>>>>>>>>>>>>
>>>>>>>>>>>> thanks,
>>>>>>>>>>>> -marina
>>>>>>>>>>>>
>>>>>>>>>>>> Sahoo wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Marina,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for sending a pointer to the server.log. In a
>>>>>>>>>>>>> nutshell, there is something wrong with your build. Read
>>>>>>>>>>>>> on, it's interesting...
>>>>>>>>>>>>>
>>>>>>>>>>>>> The reason for the linkage error is that security module is
>>>>>>>>>>>>> able to see javax.servlet.http.HttpServletRequestc.class
>>>>>>>>>>>>> loader by two class loaders. It is a classic case of
>>>>>>>>>>>>> constraint violation during class loading as explained below:
>>>>>>>>>>>>>
>>>>>>>>>>>>> com.sun.enterprise.security.web.integration.WebSecurityManager
>>>>>>>>>>>>> calls
>>>>>>>>>>>>> javax.security.jacc.WebUserDataPermission(javax.servlet.http.HttpServletRequest).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> This implies that /javax.servlet.http.HttpServletRequest as
>>>>>>>>>>>>> seen by
>>>>>>>>>>>>> com.sun.enterprise.security.web.integration.WebSecurityManager
>>>>>>>>>>>>> must be *same* as javax.servlet.http.HttpServletRequest as
>>>>>>>>>>>>> seen by javax.security.jacc.WebUserDataPermission/. By
>>>>>>>>>>>>> same, I mean the class name must match as well the class
>>>>>>>>>>>>> loader that defines the class.
>>>>>>>>>>>>>
>>>>>>>>>>>>> There is something going *wrong* with your build. In your
>>>>>>>>>>>>> installation, for whatever reason, I see
>>>>>>>>>>>>> *javax.security.jacc-10.0-SNAPSHOT.jar* contains
>>>>>>>>>>>>> javax.servlet.http.HttpServletRequest.class in addition to
>>>>>>>>>>>>> many other javax.servlet.http classes. For that reason,
>>>>>>>>>>>>> javax.security.jacc-10.0-SNAPSHOT.jar does not import (OSGi
>>>>>>>>>>>>> Import-Package) javax.servlet.http package. As a result,
>>>>>>>>>>>>> javax.security.jacc module is loading its own version of
>>>>>>>>>>>>> javax.servlet.http.HttpServletRequest.class, where as
>>>>>>>>>>>>> com.sun.enterprise.security.web.integration.WebSecurityManager
>>>>>>>>>>>>> is using the class as loaded by javax.servlet module. Hence
>>>>>>>>>>>>> the constraint violation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This does *not* happen for us, because in our environment,
>>>>>>>>>>>>> javax.servlet.http classes are part of javax.servlet module
>>>>>>>>>>>>> only. So, we have to investigate why your build (and
>>>>>>>>>>>>> occasionally hudson build) produces such
>>>>>>>>>>>>> javax.security.jacc module. I suggest you do the following:
>>>>>>>>>>>>>
>>>>>>>>>>>>> mvn -X -f v3/security/javax.security.jacc/pom.xml clean
>>>>>>>>>>>>> install > bld.log 2>&1
>>>>>>>>>>>>>
>>>>>>>>>>>>> If this does not produce, try doing a build from v3 level
>>>>>>>>>>>>> with -X option and see if produces such a bad jar. if yes,
>>>>>>>>>>>>> then send me the bld.log as well as mvn version you use.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Sahoo
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sahoo wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Marina,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This is good in some other sense. Kumar (from security
>>>>>>>>>>>>>> team) was telling me that he has seen those errors
>>>>>>>>>>>>>> occasionally in hudson QL job, but could never reproduce
>>>>>>>>>>>>>> in local enviriment to get to the root cause. If you have
>>>>>>>>>>>>>> the environment, please change felix.log.level to 4 in
>>>>>>>>>>>>>> felix/conf/config.properties, run the test and send the
>>>>>>>>>>>>>> log to Kumar and me to take a look. He had sent a mail to
>>>>>>>>>>>>>> this effect to dev@ a few days back.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> Sahoo
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Marina Vatkina wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'm seeing this stack trace in the log. Can it be related?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>>> -marina
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [#|2008-09-12T17:15:23.796-0700|SEVERE|GlassFish10.0|org.apache.catalina.connector.CoyoteAdapter|_ThreadID=21;_T
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> hreadName=Thread-3;|PWC3989: An exception or error
>>>>>>>>>>>>>>> occurred in the container during the request processing
>>>>>>>>>>>>>>> java.lang.LinkageError: loader constraints violated when
>>>>>>>>>>>>>>> linking javax/servlet/http/HttpServletRequest class
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.enterprise.security.web.integration.WebSecurityManager.hasUserDataPermission(WebSecurityManag
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> er.java:505)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.web.security.RealmAdapter.hasUserDataPermission(RealmAdapter.java:858)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:574)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:687)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:657)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:96)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 98)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:187)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:719)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:657)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:651)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1030)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:142)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:719)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:657)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:651)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1030)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:307)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:178)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.http.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:621)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.http.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:552)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.http.DefaultProcessorTask.process(DefaultProcessorTask.java:800)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:152)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.enterprise.v3.services.impl.GlassfishProtocolChain.executeProtocolFilter(GlassfishProtocolCha
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> in.java:70)
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:103)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:89)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:309)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>> com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:168)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Marina Vatkina wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've just checked out and rebuild my ws.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> -marina
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] Connecting to:
>>>>>>>>>>>>>>>> http://localhost:8080/hellojspsecure/first.html
>>>>>>>>>>>>>>>> [testng] ERROR: Anonymous Client access was Allowed,
>>>>>>>>>>>>>>>> security-constraint not Enforced correctly
>>>>>>>>>>>>>>>> [testng] FAILED: simpleJSPTestPage
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: Anonymous Client
>>>>>>>>>>>>>>>> acess was Allowed, security-constraint not Enforced
>>>>>>>>>>>>>>>> expected:<true> but was:<false>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.hellojsp.HelloSecurityTestNG.simpleJSPTestPage(HelloSecurityTestNG.java:91)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>>>>>>>>>> [testng] FAILED: simpleServletTest
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: Anonymous Client
>>>>>>>>>>>>>>>> acess was Allowed, security-constraint not Enforced
>>>>>>>>>>>>>>>> expected:<true> but was:<false>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.hellojsp.HelloSecurityTestNG.simpleServletTest(HelloSecurityTestNG.java:219)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>>>>>>>>>> [testng] FAILED: staticHTMLPageTest
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: Anonymous Client
>>>>>>>>>>>>>>>> acess was Allowed, security-constraint not Enforced
>>>>>>>>>>>>>>>> expected:<true> but was:<false>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.hellojsp.HelloSecurityTestNG.staticHTMLPageTest(HelloSecurityTestNG.java:172)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ===============================================
>>>>>>>>>>>>>>>> [testng] security_hello_jsp
>>>>>>>>>>>>>>>> [testng] Tests run: 3, Failures: 3, Skips: 0
>>>>>>>>>>>>>>>> [testng] ===============================================
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] BASIC auth: Group mapped user, testuser42
>>>>>>>>>>>>>>>> [testng] String not found: RESULT: principal: testuser42
>>>>>>>>>>>>>>>> [testng] BASIC auth: Not authorized user, testuser42
>>>>>>>>>>>>>>>> [testng] String not found: HTTP/1.1 403
>>>>>>>>>>>>>>>> [testng] BASIC auth: Valid user and invalid password
>>>>>>>>>>>>>>>> [testng] String not found: HTTP/1.1 401
>>>>>>>>>>>>>>>> [testng] BASIC auth: Role Mapped User, testuser3
>>>>>>>>>>>>>>>> [testng] String not found: RESULT: principal: testuser3
>>>>>>>>>>>>>>>> [testng] FAILED: testAuthGroupMappedUser
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: BASIC auth: Group
>>>>>>>>>>>>>>>> mapped user, testuser42 expected:<false> but was:<true>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.basicauth.BasicAuthTestNG.testAuthGroupMappedUser(BasicAuthTestNG.java:106)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 25 stack frames
>>>>>>>>>>>>>>>> [testng] FAILED: testAuthNotAuthorizedUser
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: BASIC auth: Not
>>>>>>>>>>>>>>>> authorized user, testuser42 expected:<false> but was:<true>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.basicauth.BasicAuthTestNG.testAuthNotAuthorizedUser(BasicAuthTestNG.java:124)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 25 stack frames
>>>>>>>>>>>>>>>> [testng] FAILED: testAuthNotValidPassword
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: BASIC auth: Valid
>>>>>>>>>>>>>>>> user and invalid password expected:<false> but was:<true>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.basicauth.BasicAuthTestNG.testAuthNotValidPassword(BasicAuthTestNG.java:142)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 25 stack frames
>>>>>>>>>>>>>>>> [testng] FAILED: testAuthRoleMappedUser
>>>>>>>>>>>>>>>> [testng] java.lang.AssertionError: BASIC auth: Role
>>>>>>>>>>>>>>>> Mapped User, testuser3 expected:<false> but was:<true>
>>>>>>>>>>>>>>>> [testng] at
>>>>>>>>>>>>>>>> test.security.basicauth.BasicAuthTestNG.testAuthRoleMappedUser(BasicAuthTestNG.java:88)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ... Removed 25 stack frames
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [testng] ===============================================
>>>>>>>>>>>>>>>> [testng] security_hello_basicauth
>>>>>>>>>>>>>>>> [testng] Tests run: 4, Failures: 4, Skips: 0
>>>>>>>>>>>>>>>> [testng] ===============================================
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>>> dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>>> dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>>>>> dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>
>>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>
>>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>>>> dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>
>>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>