Didn't realize Tim's reply had been to the alias.
Therefore, forwarding my reply to him to the alias.
Jan
attached mail follows:
Hi Tim,
Tim Quinn wrote On 04/19/07 11:45 AM,:
> Hi, Jan.
>
> Really great detective work on this.
Thanks!
> I probably should have followed this more closely given that the
> serialization logic I added was involved.
>
> In your research into this, did you see any place that the
> serialization code could have protected better against something like
> this?
>
> Because the serialization (and deserialization) of descriptors was a
> performance-only enhancement, I wrote the SerializedDescriptorHelper
> class to catch all Throwables that emerge from the attempt to
> serialize. (Although without the serialized descriptor the next
> reload of the app would take longer, an exception during serialization
> would cause no functional problem.) The JavaDoc for ThreadDeath says
> that this type of error should be rethrown - I guess to actually make
> sure the thread does stop. But as written the logic does not do that
> - it logs the warning with the stack trace and continues.
>
> In this case do you think it would be better if the code caught
> Exception rather than Throwable, which would let ThreadDeath and other
> descendants of Error (as opposed to Exception) bypass the handling and
> "float" to the top?
I think your logic does the right thing as it stands.
I really think Tomcat should have picked an error (or exception) other
than ThreadDeath
to signal when a webapp classloader that has been deactivated is asked
to load a
resource. It's not that any thread was stopped in this scenario, only
the webapp that
was executing on the thread was stopped (after which the thread was
returned to its
thread pool).
Jan
>
> Thanks.
>
> - Tim
>
> Jan Luehe wrote:
>
>> Ashish,
>>
>> Ashish Sahni wrote On 04/03/07 04:45 PM,:
>>
>>> Hi Jeanfrancois,
>>>
>>> First to clarify since you mention 'undeployed' - maybe you're
>>> implying implicit undeployment
>>> that might be happening during re-deployment but in any case this is
>>> what I do:
>>> 1. Deploy app
>>> 2. Request app services
>>> 3. Deploy (same) app (again) ie no explicit undeployment
>>>
>>> Secondly, I'm not sure if application is 'holding the classloader'
>>> since the application uses
>>> the jruby runtime libraries. I'm cc'ing Charles and Thomas, the
>>> jruby developers how might
>>> be able to shed light on your question.
>>
>>
>>
>> I've investigated this issue further, and it indeed turns out to be an
>> issue with jruby.
>>
>> The java.lang.ThreadDeath is thrown during redeployment of
>> depot_l2.war, when the deployment code attempts to serialize the
>> webapp's AppDescriptor, and in doing so, needs to acquire a
>> MessageDigest
>> instance for computing a serialization UID:
>>
>> java.lang.ThreadDeath
>> at
>> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1325)
>>
>> ...
>> at java.lang.Class.newInstance(Class.java:303)
>> at java.security.Provider$Service.newInstance(Provider.java:1130)
>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:220)
>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:147)
>> at java.security.Security.getImpl(Security.java:658)
>> at java.security.MessageDigest.getInstance(MessageDigest.java:122)
>> at
>> java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1731)
>>
>> ...
>> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
>> at
>> com.sun.enterprise.instance.SerializedDescriptorHelper.storeSerializedDescriptor(SerializedDescriptorHelper.java:239)
>>
>> at
>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:143)
>>
>> at
>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:123)
>>
>> at
>> com.sun.enterprise.instance.BaseManager.saveAppDescriptor(BaseManager.java:684)
>>
>> ...
>> at
>> com.sun.enterprise.management.deploy.DeployThread.deploy(DeployThread.java:174)
>>
>> at
>> com.sun.enterprise.management.deploy.DeployThread.run(DeployThread.java:210)
>>
>>
>>
>> It turns out that jruby registers its own provider for cryptographic
>> services (named "BC"), but never unregisters it during undeployment,
>> causing the subsequent redeployment code to attempt to grab a
>> MessageDigest instance whose classloader has been deactivated, which
>> in turn has the deactivated classloader (an instance of
>> WebappClassLoader) throw the ThreadDeath with the above stacktrace.
>>
>> I've instrumented java.security.Security.addProvider() and
>> removeProvider() to print the caller's stacktrace. I see
>> Security.addProvider() getting invoked when accessing
>> http://<host>:<port>/depot_l2/store/, see attached stack trace, but I
>> don't see any invocation of Security.removeProvider during
>> undeployment.
>>
>> jruby needs to unregister its provider for cryptographic services
>> during undeployment, by calling
>> java.security.Security.removeProvider("BC").
>>
>> This could be done inside
>> org.jruby.webapp.AbstractRailsServlet.destroy().
>>
>>
>> Jan
>>
>>>
>>> Thanks
>>> Ashish
>>>
>>>
>>> Jeanfrancois Arcand wrote:
>>>
>>>> Hi Ashish,
>>>>
>>>> there is two possibilities:
>>>>
>>>> (1) There is a thread pool or object in GlassFish that store the
>>>> ContextClassloader and try to re-use it. The ContextClassLoader of
>>>> a web application is always a WebappClassloader, and that
>>>> classloader is marked dead when the application is undeployed. In
>>>> 9.0, we fixed a bug in the ORB thread pool related to that problem.
>>>> This time I suspect the component that create it is:
>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.storeSerializedDescriptor(SerializedDescriptorHelper.java:239)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:143)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:123)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.BaseManager.saveAppDescriptor(BaseManager.java:684)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.EJBCompiler.preDeployModule(EJBCompiler.java:352)
>>>>>
>>>>
>>>>
>>>>
>>>> (2) Your application is holding the classloader. Are you loading
>>>> classes using the ContextClassloader and try to re-use that
>>>> Classloader?
>>>>
>>>> If your application doesn't use or store the ContextClassLoader,
>>>> that means one component in GlassFish is doing it. Now to find the
>>>> component that does it is far from simple. Last time I've
>>>> instrumented the JDK classes to find which component was doing
>>>> that. Just file a bug if you think your application is not storing
>>>> the classloader.
>>>>
>>>> Thanks
>>>>
>>>> -- Jeanfrancois
>>>>
>>>>
>>>> Ashish Sahni wrote:
>>>>
>>>>> Hi All,
>>>>> I've been seeing this error(see attached mail) consistently for
>>>>> the last few days.
>>>>> It can easily reproduced on re-deployment. If interested I can
>>>>> send you the
>>>>> webapp used - its quite large. I have not checked if there's a bug
>>>>> filed
>>>>> on this issue already. Let me know if you have any thoughts.
>>>>>
>>>>> Thanks
>>>>> Ashish
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> Subject:
>>>>> deployment error/thread death
>>>>> From:
>>>>> ashish_at_hysteria.SFBay.Sun.COM
>>>>> Date:
>>>>> Tue, 03 Apr 2007 15:11:55 -0700 (PDT)
>>>>> To:
>>>>> Ashish.Sahni_at_sun.com
>>>>>
>>>>> To:
>>>>> Ashish.Sahni_at_sun.com
>>>>>
>>>>>
>>>>> [#|2007-04-03T15:08:22.618-0700|INFO|sun-appserver9.1|org.apache.catalina.loader.WebappClassLoader|_ThreadID=17;_ThreadName=Thread-29;|PWC1635:
>>>>> Illegal access: this web application instance has been stopped
>>>>> already (the eventual following stack trace is caused by an error
>>>>> thrown for debugging purposes as well as to attempt to terminate
>>>>> the thread which caused the illegal access, and has no functional
>>>>> impact)|#]
>>>>>
>>>>> [#|2007-04-03T15:08:22.618-0700|INFO|sun-appserver9.1|org.apache.catalina.loader.WebappClassLoader|_ThreadID=17;_ThreadName=Thread-29;|PWC1635:
>>>>> Illegal access: this web application instance has been stopped
>>>>> already (the eventual following stack trace is caused by an error
>>>>> thrown for debugging purposes as well as to attempt to terminate
>>>>> the thread which caused the illegal access, and has no functional
>>>>> impact)|#]
>>>>>
>>>>> [#|2007-04-03T15:08:22.628-0700|WARNING|sun-appserver9.1|javax.enterprise.system.core|_ThreadID=17;_ThreadName=Thread-29;_RequestID=b4d48310-63bf-4ab9-95ac-99d72a398625;|CORE5064:
>>>>> Error serializing descriptor for application cookbook-test-1.2.2;
>>>>> continuing
>>>>> java.lang.ThreadDeath
>>>>> at
>>>>> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1325)
>>>>>
>>>>> at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>>>>> at sun.misc.Unsafe.defineClass(Native Method)
>>>>> at sun.reflect.ClassDefiner.defineClass(ClassDefiner.java:45)
>>>>> at
>>>>> sun.reflect.MethodAccessorGenerator$1.run(MethodAccessorGenerator.java:381)
>>>>>
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at
>>>>> sun.reflect.MethodAccessorGenerator.generate(MethodAccessorGenerator.java:377)
>>>>>
>>>>> at
>>>>> sun.reflect.MethodAccessorGenerator.generateConstructor(MethodAccessorGenerator.java:76)
>>>>>
>>>>> at
>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:30)
>>>>>
>>>>> at
>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>>>>>
>>>>> at
>>>>> java.lang.reflect.Constructor.newInstance(Constructor.java:494)
>>>>> at java.lang.Class.newInstance0(Class.java:350)
>>>>> at java.lang.Class.newInstance(Class.java:303)
>>>>> at java.security.Provider$Service.newInstance(Provider.java:1130)
>>>>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:220)
>>>>> at sun.security.jca.GetInstance.getInstance(GetInstance.java:147)
>>>>> at java.security.Security.getImpl(Security.java:658)
>>>>> at
>>>>> java.security.MessageDigest.getInstance(MessageDigest.java:122)
>>>>> at
>>>>> java.io.ObjectStreamClass.computeDefaultSUID(ObjectStreamClass.java:1731)
>>>>>
>>>>> at
>>>>> java.io.ObjectStreamClass.access$100(ObjectStreamClass.java:52)
>>>>> at java.io.ObjectStreamClass$1.run(ObjectStreamClass.java:192)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at
>>>>> java.io.ObjectStreamClass.getSerialVersionUID(ObjectStreamClass.java:189)
>>>>>
>>>>> at
>>>>> java.io.ObjectStreamClass.writeNonProxy(ObjectStreamClass.java:651)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeClassDescriptor(ObjectOutputStream.java:615)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1169)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1127)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1178)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1127)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1284)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
>>>>> at
>>>>> com.sun.enterprise.deployment.util.ModuleDescriptor.writeObject(ModuleDescriptor.java:242)
>>>>>
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>
>>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>>> at
>>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
>>>>> at java.util.HashSet.writeObject(HashSet.java:254)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>
>>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>>> at
>>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
>>>>> at
>>>>> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1375)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1347)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1079)
>>>>> at
>>>>> java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:302)
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.storeSerializedDescriptor(SerializedDescriptorHelper.java:239)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:143)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.SerializedDescriptorHelper.store(SerializedDescriptorHelper.java:123)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.instance.BaseManager.saveAppDescriptor(BaseManager.java:684)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.EJBCompiler.preDeployModule(EJBCompiler.java:352)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.EJBCompiler.compile(EJBCompiler.java:217)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.ModuleDeployer.runEJBC(ModuleDeployer.java:801)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.WebModuleDeployer.deploy(WebModuleDeployer.java:168)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.backend.ModuleDeployer.doRequestFinish(ModuleDeployer.java:166)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.phasing.J2EECPhase.runPhase(J2EECPhase.java:174)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.phasing.DeploymentPhase.executePhase(DeploymentPhase.java:95)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.phasing.PEDeploymentService.executePhases(PEDeploymentService.java:892)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.phasing.PEDeploymentService.deploy(PEDeploymentService.java:266)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.deployment.phasing.PEDeploymentService.deploy(PEDeploymentService.java:761)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.management.deploy.DeployThread.deploy(DeployThread.java:174)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.management.deploy.DeployThread.run(DeployThread.java:210)
>>>>>
>>>>> |#]
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>------------------------------------------------------------------------
>>
>> at java.lang.Thread.dumpStack(Thread.java:1158)
>> at java.security.Security.insertProviderAt(Security.java:329)
>> at org.jruby.RubyDigest.createDigest(RubyDigest.java:50)
>> at org.jruby.libraries.DigestLibrary.load(DigestLibrary.java:40)
>> at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:277)
>> at org.jruby.runtime.load.LoadService.require(LoadService.java:299)
>> at org.jruby.RubyDigest.createDigestSHA2(RubyDigest.java:104)
>> at org.jruby.libraries.DigestLibrary$SHA2.load(DigestLibrary.java:63)
>> at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:277)
>> at org.jruby.runtime.load.LoadService.require(LoadService.java:299)
>> at org.jruby.RubyKernel.require(RubyKernel.java:670)
>> at org.jruby.RubyKernelInvokerSrequire1.call(Unknown Source)
>> at org.jruby.runtime.callback.InvocationCallback.execute(InvocationCallback.java:49)
>> at org.jruby.internal.runtime.methods.FullFunctionCallbackMethod.internalCall(FullFunctionCallbackMethod.java:76)
>> at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:69)
>> at org.jruby.RubyObject.callMethod(RubyObject.java:487)
>> at org.jruby.RubyObject.callMethod(RubyObject.java:395)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:770)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:298)
>> at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:163)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:1317)
>> at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:163)
>> at org.jruby.RubyObject.eval(RubyObject.java:563)
>> at org.jruby.Ruby.loadScript(Ruby.java:1179)
>> at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:53)
>> at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:277)
>> at org.jruby.runtime.load.LoadService.require(LoadService.java:299)
>> at org.jruby.RubyKernel.require(RubyKernel.java:670)
>> at org.jruby.RubyKernelInvokerSrequire1.call(Unknown Source)
>> at org.jruby.runtime.callback.InvocationCallback.execute(InvocationCallback.java:49)
>> at org.jruby.internal.runtime.methods.FullFunctionCallbackMethod.internalCall(FullFunctionCallbackMethod.java:76)
>> at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:69)
>> at org.jruby.RubyObject.callMethod(RubyObject.java:487)
>> at org.jruby.RubyObject.callMethod(RubyObject.java:395)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:770)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:298)
>> at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:163)
>> at org.jruby.evaluator.EvaluationState.evalInternal(EvaluationState.java:1317)
>> at org.jruby.evaluator.EvaluationState.eval(EvaluationState.java:163)
>> at org.jruby.RubyObject.eval(RubyObject.java:563)
>> at org.jruby.Ruby.loadScript(Ruby.java:1179)
>> at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:53)
>> at org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:277)
>> at org.jruby.runtime.load.LoadService.require(LoadService.java:299)
>> at org.jruby.webapp.RailsFactory.makeObject(RailsFactory.java:109)
>> at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:840)
>> at org.jruby.webapp.AbstractRailsServlet.serviceRequest(AbstractRailsServlet.java:149)
>> at org.jruby.webapp.AbstractRailsServlet.service(AbstractRailsServlet.java:136)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:818)
>> at org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:398)
>> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
>> at org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:258)
>> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:189)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:611)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:564)
>> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:81)
>> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:193)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:611)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:564)
>> at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:558)
>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1067)
>> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:138)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:611)
>> at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:564)
>> at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:558)
>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1067)
>> at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:255)
>> at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:618)
>> at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:549)
>> at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:790)
>> at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:326)
>> at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:248)
>> at com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:199)
>> at com.sun.enterprise.web.portunif.PortUnificationPipeline$PUTask.doTask(PortUnificationPipeline.java:345)
>> at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:252)
>> at com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:93)
>>
>>
>>
>>------------------------------------------------------------------------
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>>