Thanks Hong for your fast reply and clear analysis, this is the best way
to appreciate Matthias's effort, what a shinning star :-)
Judy
Hong Zhang wrote:
> Hi, Matthias
> Thanks so much for testing with the latest build and glad to hear
> that you have it worked after expanding the ejb jar!
> A couple comments:
> 1. I agree with your assesement that the EOF error probably due to
> Grizzly Thread time out rather than there is problem with web.xml (and
> the success of your later experiment further proves it).
> I will try to find out where/how we can configure this time out
> thing so we can try it again with the non-expanded version to see what
> happens. Jeanfrancois is travelling this week but I will talk to other
> people in the grizzly team to see if they know.
> 2. I will also look into why it makes such a big difference if the
> ejb jar is expanded. The logic to expand the archive is very similar
> to what we had in v2, so there might be other side effects of having a
> jar versus directory here (cannot think of any yet). In your email,
> you said the client times out after 10 minutes, can you collect the
> jstack trace say at 5 minutes, 8 minutes to see what work the thread
> is busy doing, that might give us some ideas where to look.
>
> As today is the HCF date, I am not sure how much further
> improvement we can get into the final build of v3 for this. We will
> try but I just want to confirm that with the workaround of expanding
> the ejb jar, it's no longer a show stopper for you?
>
> Thanks,
>
> - Hong
>
> matthias.fraass_at_tricoder.net wrote:
>> Update2:
>>
>> Success!
>> I deflatet the ejb.jar to ejb_jar and packaged it this way in the EAR
>> (changed application.xml, of course).
>> -> deployment success (some errors, but they don't have to do with
>> this problem) after 6 minutes!
>>
>> Could you please check that EOF error, anyway?
>>
>> -Matthias
>>
>>
>> 2009/11/9 matthias.fraass_at_tricoder.net <matthias.fraass_at_gmail.com>:
>>
>>> Update:
>>>
>>> after 15mins I got an exception.
>>>
>>> I'm not sure about this. The web.xml is fine (deploys in GF v2).
>>> Could it be that some monitoring thread (GrizzlyServiceListener )
>>> decided that the deployment thread was idling and killed it? And that
>>> the EOF error was just a symptom for this, and not the cause of the
>>> deployment failure?
>>>
>>> I started the Deployment 08:23 and the asadmin client timed out at 09:33.
>>> Five minutes later:
>>>
>>> [#|2009-11-09T08:38:26.390+0100|WARNING|glassfishv3.0|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=18;_ThreadName=Thread-1;|Interrupting
>>> idle Thread: http-thread-pool-4848-(2)|#]
>>>
>>> [#|2009-11-09T08:38:27.953+0100|SEVERE|glassfishv3.0|javax.enterprise.system.tools.deployment.org.glassfish.deployment.common|_ThreadID=26;_ThreadName=Thread-1;|DPL8015:
>>> Invalid Deployment Descriptors in Deployment descriptor file
>>> WEB-INF/web.xml in archive [xyz_war].
>>> Line 1 Column 1 -- Premature end of file.|#]
>>>
>>> [#|2009-11-09T08:38:27.953+0100|SEVERE|glassfishv3.0|javax.enterprise.system.tools.deployment.org.glassfish.deployment.common|_ThreadID=26;_ThreadName=Thread-1;|DPL8005:
>>> Deployment Descriptor parsing failure : Premature end of file.|#]
>>>
>>> [#|2009-11-09T08:38:27.953+0100|SEVERE|glassfishv3.0|javax.enterprise.system.core.com.sun.enterprise.v3.server|_ThreadID=26;_ThreadName=Thread-1;|Exception
>>> while deploying the app
>>> java.io.IOException: org.xml.sax.SAXParseException: Premature end of file.
>>> at org.glassfish.javaee.core.deployment.DolProvider.load(DolProvider.java:111)
>>> at org.glassfish.javaee.core.deployment.DolProvider.load(DolProvider.java:41)
>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.loadDeployer(ApplicationLifecycle.java:610)
>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.setupContainerInfos(ApplicationLifecycle.java:552)
>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:260)
>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLifecycle.java:183)
>>> at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:272)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:320)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1176)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$900(CommandRunnerImpl.java:83)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1235)
>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1224)
>>> at com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:362)
>>> at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:201)
>>> at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:166)
>>> at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:100)
>>> at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:245)
>>> at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:789)
>>> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:697)
>>> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:952)
>>> at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:166)
>>> at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
>>> at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
>>> at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>>> at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>>> at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
>>> at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>>> at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>>> at java.lang.Thread.run(Thread.java:619)
>>> Caused by: org.xml.sax.SAXParseException: Premature end of file.
>>> at com.sun.enterprise.deployment.io.DeploymentDescriptorFile.read(DeploymentDescriptorFile.java:304)
>>> at com.sun.enterprise.deployment.io.DeploymentDescriptorFile.read(DeploymentDescriptorFile.java:225)
>>> at com.sun.enterprise.deployment.archivist.Archivist.readStandardDeploymentDescriptor(Archivist.java:614)
>>> at com.sun.enterprise.deployment.archivist.Archivist.readDeploymentDescriptors(Archivist.java:366)
>>> at com.sun.enterprise.deployment.archivist.Archivist.open(Archivist.java:238)
>>> at com.sun.enterprise.deployment.archivist.Archivist.open(Archivist.java:247)
>>> at com.sun.enterprise.deployment.archivist.ApplicationArchivist.readModulesDescriptors(ApplicationArchivist.java:608)
>>> at com.sun.enterprise.deployment.archivist.ApplicationArchivist.openWith(ApplicationArchivist.java:244)
>>> at com.sun.enterprise.deployment.archivist.ApplicationFactory.openWith(ApplicationFactory.java:222)
>>> at org.glassfish.javaee.core.deployment.DolProvider.load(DolProvider.java:108)
>>> ... 31 more
>>> Caused by: org.xml.sax.SAXParseException: Premature end of file.
>>> at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:195)
>>> at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:174)
>>> at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:388)
>>> at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1414)
>>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:1059)
>>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
>>> at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:140)
>>> at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
>>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:807)
>>> at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
>>> at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:107)
>>> at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
>>> at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
>>> at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
>>> at com.sun.enterprise.deployment.io.DeploymentDescriptorFile.read(DeploymentDescriptorFile.java:298)
>>> ... 40 more
>>> |#]
>>>
>>> [#|2009-11-09T08:38:29.750+0100|SEVERE|glassfishv3.0|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=26;_ThreadName=Thread-1;|service
>>> exception
>>> java.lang.RuntimeException: ClientAbortException:
>>> java.nio.channels.ClosedChannelException
>>> at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:233)
>>> at com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:166)
>>> at com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:100)
>>> at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:245)
>>> at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:789)
>>> at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:697)
>>> at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:952)
>>> at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:166)
>>> at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>>> at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
>>> at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
>>> at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>>> at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>>> at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
>>> at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>>> at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>>> at java.lang.Thread.run(Thread.java:619)
>>> Caused by: ClientAbortException: java.nio.channels.ClosedChannelException
>>> at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.doFlush(GrizzlyOutputBuffer.java:436)
>>> at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.flush(GrizzlyOutputBuffer.java:402)
>>> at com.sun.grizzly.tcp.http11.GrizzlyOutputStream.flush(GrizzlyOutputStream.java:138)
>>> at com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:230)
>>> ... 17 more
>>> Caused by: java.nio.channels.ClosedChannelException
>>> at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:126)
>>> at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
>>> at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:106)
>>> at com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:74)
>>> at com.sun.grizzly.http.SocketChannelOutputBuffer.flushChannel(SocketChannelOutputBuffer.java:321)
>>> at com.sun.grizzly.http.SocketChannelOutputBuffer.flushBuffer(SocketChannelOutputBuffer.java:386)
>>> at com.sun.grizzly.http.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:364)
>>> at com.sun.grizzly.http.ProcessorTask.action(ProcessorTask.java:1172)
>>> at com.sun.grizzly.tcp.Response.action(Response.java:260)
>>> at com.sun.grizzly.tcp.http11.GrizzlyOutputBuffer.doFlush(GrizzlyOutputBuffer.java:431)
>>> ... 20 more
>>> |#]
>>>
>>> The good thing might be that Friday I had to abort the deployment
>>> after 3 hours. Today it's past the EAR inflating within 15 minutes.
>>> Big improvement!
>>>
>>> Any ideas on the deployment failure, though?
>>>
>>> -Matthias
>>>
>>>
>>> 2009/11/9 matthias.fraass_at_tricoder.net <matthias.fraass_at_gmail.com>:
>>>
>>>> Hi all,
>>>>
>>>> first thing in the morning I tested glassfish-v3-b72-11_08_2009.zip.
>>>>
>>>> It's still timing out :(.
>>>> The stacktrace is a bit different, though - the
>>>> GenericAnnotationDetector is gone:
>>>>
>>>> "http-thread-pool-4848-(2)" daemon prio=10 tid=0x2f649400 nid=0x1a0
>>>> runnable [0x3155e000]
>>>> java.lang.Thread.State: RUNNABLE
>>>> at java.util.zip.Inflater.inflateBytes(Native Method)
>>>> at java.util.zip.Inflater.inflate(Inflater.java:223)
>>>> - locked <0x02c68600> (a java.util.zip.Inflater)
>>>> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:135)
>>>> at java.util.zip.ZipInputStream.read(ZipInputStream.java:146)
>>>> at java.util.jar.JarInputStream.read(JarInputStream.java:177)
>>>> at java.util.zip.ZipInputStream.closeEntry(ZipInputStream.java:92)
>>>> at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:70)
>>>> at java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:118)
>>>> at java.util.jar.JarInputStream.getNextJarEntry(JarInputStream.java:149)
>>>> at com.sun.enterprise.deployment.deploy.shared.InputJarArchive.getEntry(InputJarArchive.java:239)
>>>> at org.glassfish.internal.deployment.GenericHandler.expand(GenericHandler.java:46)
>>>> at org.glassfish.javaee.full.deployment.EarHandler.expand(EarHandler.java:162)
>>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.getContext(ApplicationLifecycle.java:1068)
>>>> at com.sun.enterprise.v3.server.ApplicationLifecycle.access$100(ApplicationLifecycle.java:99)
>>>> at com.sun.enterprise.v3.server.ApplicationLifecycle$DeploymentContextBuidlerImpl.build(ApplicationLifecycle.java:1004)
>>>> at org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:226)
>>>> at com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImpl.java:305)
>>>>
>>>> -Matthias
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: quality-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: quality-help_at_glassfish.dev.java.net
>>
>>
>