Can you confirm that you received the ear file?
Thanks,
-Todd
From: Cerwin, Todd
Sent: Wednesday, August 11, 2010 3:07 PM
To: 'Oleksiy.Stashok_at_Sun.COM'; users_at_grizzly.dev.java.net
Cc: tjquinn_at_dev.java.net; King, Bryan; Tim Quinn
Subject: RE: Grizzly Bug?
Ok, I hope the ear makes it over to you and you can reproduce this.
We are running on Glassfish V3.0.1, you probably need a JDBC JNDI lookup
for jdbc/gf3-starter and a mail resource for mail/gf3-starter other than
that everything should be standard.
I think I found a way to reproduce this without actually re-building the
application.
Just try to follow these steps:
1. Deploy the ear through the admin console
2. Enable Web Start
3. Launch the appclient from the application action
(gf3-starter-client-1.0-DEV.jar)
4. Close the client app
5. Select view endpoint for the "StarterWebServive" wsdl view this
a couple of times, you can see the generated wsdl end point.
6. Make a slight change to something in the .ear package, for my
test case I just added some bogus text file to the META-INF directory
and save the archive with the same name
7. Undeploy the old version of the ear
8. Deploy the new ear with the new content
9. Enable Web Start
10. Launch the same appclient
11. Close the client app
12. Now select to view the same end point a few times. It depends
when you hit the right thread, but the end point is occasionally
unavailable
Hopefully the first round of testing will reproduce my results.
Thanks,
-Todd
From: Oleksiy.Stashok_at_Sun.COM [mailto:Oleksiy.Stashok_at_Sun.COM]
Sent: Wednesday, August 11, 2010 1:19 PM
To: users_at_grizzly.dev.java.net
Cc: tjquinn_at_dev.java.net; King, Bryan; Tim Quinn; Cerwin, Todd
Subject: Re: Grizzly Bug?
Hi Todd,
is it possible for you to provide the testcase, using which we can
reproduce the problem. If you don't want to publish the code - you can
send it to me directly.
Thanks.
WBR,
Alexey.
On Aug 11, 2010, at 18:42 , Cerwin, Todd wrote:
Hello,
I was hoping someone from the Grizzly development team could help us
troubleshoot a major issue we have been finding after upgrading from
Glassfish V2 to V3. Basically we install our application as an ear
module and have all EE functionality working, then launch Web Start in
which we see certain exceptions related to closing http connections and
subsequent Web Service requests on that same http thread will then fail
until an application restart. There are more details in this thread and
I can send you any server logs that might help.
Thanks,
-Todd
Web Start Exception
[#|2010-08-05T10:30:04.401-0700|SEVERE|oracle-glassfish3.0.1|javax.enter
prise.system.container.appclient.org.glassfish.appclient.server.core.jws
|_ThreadID=37;_ThreadName=Thread-1;|Adapter[/___JWSappclient/___system]
s1as/glassfish/modules/org.eclipse.persistence.core.jar
java.io.IOException: An established connection was aborted by the
software in your host machine
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at
sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:33)
at
sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:100)
at sun.nio.ch.IOUtil.write(IOUtil.java:71)
at
sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at
com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:106)
at
com.sun.grizzly.util.OutputWriter.flushChannel(OutputWriter.java:74)
at
com.sun.grizzly.http.SocketChannelOutputBuffer.flushChannel(SocketChanne
lOutputBuffer.java:321)
at
com.sun.grizzly.http.SocketChannelOutputBuffer.flushBuffer(SocketChannel
OutputBuffer.java:392)
at
com.sun.grizzly.http.SocketChannelOutputBuffer.realWriteBytes(SocketChan
nelOutputBuffer.java:279)
at
com.sun.grizzly.tcp.http11.InternalOutputBuffer$OutputStreamOutputBuffer
.doWrite(InternalOutputBuffer.java:884)
at
com.sun.grizzly.tcp.http11.filters.IdentityOutputFilter.doWrite(Identity
OutputFilter.java:155)
at
com.sun.grizzly.tcp.http11.InternalOutputBuffer.doWrite(InternalOutputBu
ffer.java:640)
at
com.sun.grizzly.tcp.Response.doWrite(Response.java:676)
at
com.sun.grizzly.tcp.StaticResourcesAdapter.service(StaticResourcesAdapte
r.java:229)
at
org.glassfish.appclient.server.core.jws.RestrictedContentAdapter.process
Content(RestrictedContentAdapter.java:266)
at
org.glassfish.appclient.server.core.jws.RestrictedContentAdapter.service
Content(RestrictedContentAdapter.java:226)
at
org.glassfish.appclient.server.core.jws.AppClientHTTPAdapter.service(App
ClientHTTPAdapter.java:151)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
Subsequent Web Service Exceptions
[#|2010-08-05T10:30:39.135-0700|WARNING|oracle-glassfish3.0.1|javax.ente
rprise.webservices.org.glassfish.webservices|_ThreadID=37;_ThreadName=Th
read-1;|invocation error on ejb endpoint LimsAutomationRunner at
/LimsAutomationRunnerService/LimsAutomationRunner : null
java.lang.NullPointerException
at
com.sun.grizzly.tcp.http11.GrizzlyOutputStream.write(GrizzlyOutputStream
.java:128)
at
com.sun.grizzly.http.servlet.ServletOutputStreamImpl.write(ServletOutput
StreamImpl.java:84)
at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:96)
at
com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:214)
at
com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:311)
at
com.sun.xml.ws.streaming.XMLStreamWriterUtil.getOutputStream(XMLStreamWr
iterUtil.java:97)
at
com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:
315)
at
com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.j
ava:142)
at
com.sun.xml.ws.encoding.StreamSOAPCodec.encode(StreamSOAPCodec.java:109)
at
com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:27
8)
at
com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:
383)
at
com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93
)
at
com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter
.java:529)
at
com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:288)
at
com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapt
er.java:143)
at
org.glassfish.webservices.Ejb3MessageDispatcher.handlePost(Ejb3MessageDi
spatcher.java:116)
at
org.glassfish.webservices.Ejb3MessageDispatcher.invoke(Ejb3MessageDispat
cher.java:87)
at
org.glassfish.webservices.EjbWebServiceServlet.dispatchToEjbEndpoint(Ejb
WebServiceServlet.java:196)
at
org.glassfish.webservices.EjbWebServiceServlet.service(EjbWebServiceServ
let.java:127)
at
javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
com.sun.grizzly.http.servlet.FilterChainImpl.doFilter(FilterChainImpl.ja
va:195)
at
com.sun.grizzly.http.servlet.FilterChainImpl.invokeFilterChain(FilterCha
inImpl.java:139)
at
com.sun.grizzly.http.servlet.ServletAdapter.doService(ServletAdapter.jav
a:376)
at
com.sun.grizzly.http.servlet.ServletAdapter.service(ServletAdapter.java:
329)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
From: Tim Quinn [mailto:tim.quinn_at_oracle.com]
Sent: Tuesday, August 10, 2010 5:55 PM
To: Cerwin, Todd
Cc: tjquinn_at_dev.java.net; King, Bryan
Subject: Re: Server Logs
Todd,
I have been swamped lately. I'm sorry for the delay in responding.
The Java Web Start support in GlassFish relies on Grizzly (as you know)
and I have not worked on Grizzly's internals.
I think at this point the right place for the discussion is the grizzly
user's list users_at_grizzly.dev.java.net
It would probably be best if you posted the question there yourself if
you don't mind. I know it must feel a little like the run-around, and I
don't intend it that way.
I had thought that this was another manifestation of something I've seen
before (I think I mentioned it earlier in our conversation but I'm not
sure) but in those cases the Grizzly messages were "broken pipe"
messages, meaning the remote client (Java Web Start) broke off the
transfer. This looks different from that, especially considering the
subsequent web services errors you are seeing.
Thanks for your patience.
- Tim
On Aug 6, 2010, at 10:57 AM, Cerwin, Todd wrote:
Tim,
I made efforts to clean up those FileNotFound exceptions on my
application deploy. I realized that most of the errors were related to
the web module libraries getting compiled into the web-inf\lib folder,
so I went ahead and changed all of those to provided resources and
cleared up all exceptions except the one stack trace below. I found
there were two different versions of the xstream jar referenced (1.2 and
1.2.2), and only version 1.2 made it over into the ear\lib folder.
After removing reference to version 1.2, that exception is now gone and
there are no errors on deployment.
I am finding the same HTTP thread that causes the aborted connections
during WebStart downloads will then fail for all subsequent WebService
calls.
Web Start
[#|2010-08-06T08:48:19.850-0700|SEVERE|oracle-glassfish3.0.1|javax.enter
prise.system.container.appclient.org.glassfish.appclient.server.core.jws
|_ThreadID=28;_ThreadName=Thread-1;|Adapter[/___JWSappclient/___app/omfg
-ear-1.0-DEV] lib/beansbinding-1.2.1.jar
java.io.IOException: An established connection was aborted by the
software in your host machine
Web Service
[#|2010-08-06T08:48:37.521-0700|WARNING|oracle-glassfish3.0.1|javax.ente
rprise.webservices.org.glassfish.webservices|_ThreadID=28;_ThreadName=Th
read-1;|invocation error on ejb endpoint SideWinderTask at
/SideWinderTaskService/SideWinderTask : null
java.lang.NullPointerException
Have you had a chance to look into those Grizzly related HTTP
exceptions?
Thanks,
-Todd
[#|2010-08-05T16:04:58.536-0700|WARNING|oracle-glassfish3.0.1|org.apache
.jasper.runtime.TldScanner|_ThreadID=77;_ThreadName=Thread-1;|PWC6351:
In TLD scanning, the supplied resource
file:/C:/glassfishv3/glassfish/domains/domain1/applications/omfg-ear-1.0
-DEV/xstream-1.2.2.jar does not exist
java.io.FileNotFoundException:
C:\glassfishv3\glassfish\domains\domain1\applications\omfg-ear-1.0-DEV\x
stream-1.2.2.jar (The system cannot find the file specified)
at java.util.zip.ZipFile.open(Native Method)
at java.util.zip.ZipFile.<init>(ZipFile.java:114)
at java.util.jar.JarFile.<init>(JarFile.java:135)
at java.util.jar.JarFile.<init>(JarFile.java:72)
at
sun.net.www.protocol.jar.URLJarFile.<init>(URLJarFile.java:72)
at
sun.net.www.protocol.jar.URLJarFile.getJarFile(URLJarFile.java:48)
at
sun.net.www.protocol.jar.JarFileFactory.get(JarFileFactory.java:80)
at
sun.net.www.protocol.jar.JarURLConnection.connect(JarURLConnection.java:
104)
at
sun.net.www.protocol.jar.JarURLConnection.getJarFile(JarURLConnection.ja
va:71)
at
org.apache.jasper.runtime.TldScanner.scanJar(TldScanner.java:364)
at
org.apache.jasper.runtime.TldScanner.scanJars(TldScanner.java:607)
at
org.apache.jasper.runtime.TldScanner.scanTlds(TldScanner.java:282)
at
org.apache.jasper.runtime.TldScanner.onStartup(TldScanner.java:228)
at
org.apache.catalina.core.StandardContext.callServletContainerInitializer
s(StandardContext.java:5352)
&n bsp; at
com.sun.enterprise.web.WebModule.callServletContainerInitializers(WebMod
ule.java:550)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:5263
)
at
com.sun.enterprise.web.WebModule.start(WebModule.java:499)
at
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.ja
va:928)
at
org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:912)
at
org.apache.catalina.core.StandardHost.addChild(StandardHost.java:694)
at
com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1947
)
at
com.sun.enterprise.web.WebContainer.loadWebModule(WebContainer.java:1619
)
at
com.sun.enterprise.web.WebApplication.start(WebApplication.java:90)
at
org.glassfish.internal.data.EngineRef.start(EngineRef.java:126)
at
org.glassfish.internal.data.ModuleInfo.start(ModuleInfo.java:241)
at
org.glassfish.internal.data.ApplicationInfo.start(ApplicationInfo.java:2
36)
at
com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLife
cycle.java:339)
at
com.sun.enterprise.v3.server.ApplicationLifecycle.deploy(ApplicationLife
cycle.java:183)
at
org.glassfish.deployment.admin.DeployCommand.execute(DeployCommand.java:
272)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl$1.execute(CommandRunnerImp
l.java:305)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImp
l.java:320)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImp
l.java:1176)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl.access$900(CommandRunnerIm
pl.java:83)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(C
ommandRunnerImpl.java:1235)
at
com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(C
ommandRunnerImpl.java:1224)
at
com.sun.enterprise.v3.admin.AdminAdapter.doCommand(AdminAdapter.java:365
)
at
com.sun.enterprise.v3.admin.AdminAdapter.service(AdminAdapter.java:204)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
From: Cerwin, Todd
Sent: Tuesday, July 27, 2010 2:30 PM
To: 'tjquinn_at_dev.java.net'
Subject: RE: Server Logs
Hi Tim,
My application stability did not last very long. I just launched the
client a few more times, invoked some remote EJBs, and SOAP Web Service
methods and the Web Service calls started to intermittently fail with
the exception below.
Interestingly, I attempted to restart the domain and re-deploy the
application and the Web Service calls continued to fail intermittently.
What actually did work was to bring down the domain, delete the
java-web-start contents and re-launch the application. This caused my
Web Services to run without error repeatedly, until I again launched the
client via Web Start and now they are failing again.
I wonder if some of those resources are getting corrupt...
Attached is the following server logs from the following test case:
1. Server started with a clear java-web-start folder
2. 10 Web Service calls made without any Grizzly related errors
3. Web client launched twice
4. 10 Web Service calls made with 4 out of 10 service calls
failing.
Hope you have some ideas for me to test out.
Thanks,
-Todd
[#|2010-07-27T14:07:03.222-0700|WARNING|oracle-glassfish3.0.1|javax.ente
rprise.webservices.org.glassfish.webservices|_ThreadID=43;_ThreadName=Th
read-1;|invocation error on ejb endpoint SideWinderTask at
/SideWinderTaskService/SideWinderTask : null
java.lang.NullPointerException
at
com.sun.grizzly.tcp.http11.GrizzlyOutputStream.write(GrizzlyOutputStream
.java:128)
at
com.sun.grizzly.http.servlet.ServletOutputStreamImpl.write(ServletOutput
StreamImpl.java:84)
at com.ctc.wstx.io.UTF8Writer.flush(UTF8Writer.java:96)
at
com.ctc.wstx.sw.BufferingXmlWriter.flush(BufferingXmlWriter.java:214)
at
com.ctc.wstx.sw.BaseStreamWriter.flush(BaseStreamWriter.java:311)
at
com.sun.xml.ws.streaming.XMLStreamWriterUtil.getOutputStream(XMLStreamWr
iterUtil.java:97)
at
com.sun.xml.ws.message.jaxb.JAXBMessage.writePayloadTo(JAXBMessage.java:
315)
at
com.sun.xml.ws.message.AbstractMessageImpl.writeTo(AbstractMessageImpl.j
ava:142)
at
com.sun.xml.ws.encoding.StreamSOAPCodec.encode(StreamSOAPCodec.java:109)
at
com.sun.xml.ws.encoding.SOAPBindingCodec.encode(SOAPBindingCodec.java:27
8)
at
com.sun.xml.ws.transport.http.HttpAdapter.encodePacket(HttpAdapter.java:
383)
at
com.sun.xml.ws.transport.http.HttpAdapter.access$100(HttpAdapter.java:93
)
at
com.sun.xml.ws.transport.http.HttpAdapter$HttpToolkit.handle(HttpAdapter
.java:529)
at
com.sun.xml.ws.transport.http.HttpAdapter.handle(HttpAdapter.java:288)
at
com.sun.xml.ws.transport.http.servlet.ServletAdapter.handle(ServletAdapt
er.java:143)
at
org.glassfish.webservices.Ejb3MessageDispatcher.handlePost(Ejb3MessageDi
spatcher.java:116)
at
org.glassfish.webservices.Ejb3MessageDispatcher.invoke(Ejb3MessageDispat
cher.java:87)
at
org.glassfish.webservices.EjbWebServiceServlet.dispatchToEjbEndpoint(Ejb
WebServiceServlet.java:196)
at
org.glassfish.webservices.EjbWebServiceServlet.service(EjbWebServiceServ
let.java:127)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
com.sun.grizzly.http.servlet.FilterChainImpl.doFilter(FilterChainImpl.ja
va:195)
at
com.sun.grizzly.http.servlet.FilterChainImpl.invokeFilterChain(FilterCha
inImpl.java:139)
at
com.sun.grizzly.http.servlet.ServletAdapter.doService(ServletAdapter.jav
a:376)
at
com.sun.grizzly.http.servlet.ServletAdapter.service(ServletAdapter.java:
329)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
[#|2010-07-27T14:07:03.222-0700|WARNING|oracle-glassfish3.0.1|javax.ente
rprise.webservices.org.glassfish.webservices|_ThreadID=39;_ThreadName=Th
read-1;|cannot write SOAPFault to the HTTP response
java.lang.NullPointerException
at
com.sun.grizzly.tcp.http11.GrizzlyOutputStream.write(GrizzlyOutputStream
.java:128)
at
com.sun.grizzly.http.servlet.ServletOutputStreamImpl.write(ServletOutput
StreamImpl.java:84)
at
com.sun.xml.messaging.saaj.soap.MessageImpl.writeTo(MessageImpl.java:131
9)
at
org.glassfish.webservices.WsUtil.raiseException(WsUtil.java:1359)
at
org.glassfish.webservices.Ejb3MessageDispatcher.handlePost(Ejb3MessageDi
spatcher.java:134)
at
org.glassfish.webservices.Ejb3MessageDispatcher.invoke(Ejb3MessageDispat
cher.java:87)
at
org.glassfish.webservices.EjbWebServiceServlet.dispatchToEjbEndpoint(Ejb
WebServiceServlet.java:196)
at
org.glassfish.webservices.EjbWebServiceServlet.service(EjbWebServiceServ
let.java:127)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
com.sun.grizzly.http.servlet.FilterChainImpl.doFilter(FilterChainImpl.ja
va:195)
at
com.sun.grizzly.http.servlet.FilterChainImpl.invokeFilterChain(FilterCha
inImpl.java:139)
at
com.sun.grizzly.http.servlet.ServletAdapter.doService(ServletAdapter.jav
a:376)
at
com.sun.grizzly.http.servlet.ServletAdapter.service(ServletAdapter.java:
329)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
[#|2010-07-27T14:07:03.222-0700|INFO|oracle-glassfish3.0.1|javax.enterpr
ise.system.core.security.com.sun.appserv.security|_ThreadID=39;_ThreadNa
me=Thread-1;|Authenticating User: tcerwin_at_illumina.com|#]
[#|2010-07-27T14:07:03.222-0700|WARNING|oracle-glassfish3.0.1|javax.ente
rprise.webservices.org.glassfish.webservices|_ThreadID=43;_ThreadName=Th
read-1;|cannot write SOAPFault to the HTTP response
java.lang.NullPointerException
at
com.sun.grizzly.tcp.http11.GrizzlyOutputStream.write(GrizzlyOutputStream
.java:128)
at
com.sun.grizzly.http.servlet.ServletOutputStreamImpl.write(ServletOutput
StreamImpl.java:84)
at
com.sun.xml.messaging.saaj.soap.MessageImpl.writeTo(MessageImpl.java:131
9)
at
org.glassfish.webservices.WsUtil.raiseException(WsUtil.java:1359)
at
org.glassfish.webservices.Ejb3MessageDispatcher.handlePost(Ejb3MessageDi
spatcher.java:134)
at
org.glassfish.webservices.Ejb3MessageDispatcher.invoke(Ejb3MessageDispat
cher.java:87)
at
org.glassfish.webservices.EjbWebServiceServlet.dispatchToEjbEndpoint(Ejb
WebServiceServlet.java:196)
at
org.glassfish.webservices.EjbWebServiceServlet.service(EjbWebServiceServ
let.java:127)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
at
com.sun.grizzly.http.servlet.FilterChainImpl.doFilter(FilterChainImpl.ja
va:195)
at
com.sun.grizzly.http.servlet.FilterChainImpl.invokeFilterChain(FilterCha
inImpl.java:139)
at
com.sun.grizzly.http.servlet.ServletAdapter.doService(ServletAdapter.jav
a:376)
at
com.sun.grizzly.http.servlet.ServletAdapter.service(ServletAdapter.java:
329)
at
com.sun.grizzly.tcp.http11.GrizzlyAdapter.service(GrizzlyAdapter.java:16
6)
at
com.sun.enterprise.v3.server.HK2Dispatcher.dispath(HK2Dispatcher.java:10
0)
at
com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMap
per.java:245)
at
com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
at
com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693)
at
com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954)
at
com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter
.java:170)
at
com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtoc
olChain.java:135)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:1
02)
at
com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:8
8)
at
com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76
)
at
com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask
.java:53)
at
com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.jav
a:57)
at com.sun.grizzly.ContextTask.run(ContextTask.java:69)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool
.java:330)
at
com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.ja
va:309)
at java.lang.Thread.run(Thread.java:619)
|#]
[#|2010-07-27T14:07:03.222-0700|INFO|oracle-glassfish3.0.1|javax.enterpr
ise.system.core.security.com.sun.appserv.security|_ThreadID=43;_ThreadNa
me=Thread-1;|Authenticating User: tcerwin_at_illumina.com|#]
From: Cerwin, Todd
Sent: Tuesday, July 27, 2010 11:02 AM
To: 'tjquinn_at_dev.java.net'
Subject: Server Logs
Hi Tim and thanks again for your help investigating this!
Ok, so I tried to replicate the server crash on my development
application server, which should be identical except its running on
Windows Server 2003 as compared to Windows Server 2008 and I was unable
to reproduce the server crash, but I did see a couple of SOAP exceptions
come up during the Web Start downs.
Since the log output was gigantic with this added vm argument (added to
help troubleshoot the Grizzly errors)
(-Dcom.sun.grizzly.enableSnoop=true), I decided to remove it and I was
unable to get the 2008 server to crash again, but on three Web Start
launches I did get one that failed.
The server logs I attached is from the following steps:
Start app server
Deploy application
Launch three Web Start clients
Fire a few SOAP Web Service calls
Again, it seems to be stable after removing that vm argument, but
perhaps something in the log might give you some clues as to why I
needed to delete that Web Start cache in the first place?
I will try a few more things over the course of the day to test out my
application as it has been far from stable after the V3 upgrade.
Thanks!
-Todd