Salut,
glassfish_at_javadesktop.org wrote:
> Hello Jeanfrancois
>
> our system halted again a few minutes ago.
>
> now we collected some data.
>
>> When that happens, can you grap the output of:
>> % netstat -an | grep 8009 to get the state of the connection?
> [code]
> tcp 0 0 0.0.0.0:8009 0.0.0.0:* LISTEN
> tcp 0 0 0.0.0.0:8009 0.0.0.0:* LISTEN
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58594 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58575 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58593 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58591 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58503 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58565 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58556 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58577 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58551 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58519 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58590 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58547 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58514 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58595 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58599 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58533 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58513 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58516 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58512 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58515 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58568 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58524 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58511 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58508 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58537 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58566 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58521 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58555 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58539 SYN_RECV
> tcp 0 0 127.0.0.1:8009 127.0.0.1:58531 SYN_RECV
> tcp 0 1 127.0.0.1:58597 127.0.0.1:8009 SYN_SENT
> tcp 0 1 127.0.0.1:58592 127.0.0.1:8009 SYN_SENT
> tcp 0 1 127.0.0.1:58570 127.0.0.1:8009 SYN_SENT
>
> tcp 0 0 127.0.0.1:8009 127.0.0.1:35691 VERBUNDEN
> tcp 0 0 127.0.0.1:8009 127.0.0.1:42107 VERBUNDEN
> tcp 438 0 127.0.0.1:8009 127.0.0.1:58540 VERBUNDEN
> tcp 0 0 127.0.0.1:37629 127.0.0.1:8009 VERBUNDEN
> tcp 495 0 127.0.0.1:8009 127.0.0.1:58553 VERBUNDEN
> [...]
> tcp 0 0 127.0.0.1:38667 127.0.0.1:8009 VERBUNDEN
> tcp 741 0 127.0.0.1:8009 127.0.0.1:58538 VERBUNDEN
> tcp 0 636 127.0.0.1:58524 127.0.0.1:8009 VERBUNDEN
> tcp 0 861 127.0.0.1:58521 127.0.0.1:8009 VERBUNDEN
> tcp 0 201216 127.0.0.1:8009 127.0.0.1:37080 VERBUNDEN
>
> (522x Verbunden (connected))
Thanks for the translation :-)
>
> [/code]
>
>
>> Hum...can you get a thread dump of the GF process
>> when than happens? I'm surprised GF needs to be restarted. Just do:
>> % jstack PID > thread.dump
>
> There are a couple of threads (17) running in SocketWrite. An example
> [code]
> Thread "TP-Processor1391" thread-id 90.456 thread-stateRUNNABLERunning in native
> at: java.net.SocketOutputStream.socketWrite0(Native Method)
> at: java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
> at: java.net.SocketOutputStream.write(SocketOutputStream.java:136)
> at: org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:537)
> at: org.apache.jk.common.JkInputStream.doWrite(JkInputStream.java:162)
> at: org.apache.coyote.Response.doWrite(Response.java:599)
> at: org.apache.coyote.tomcat5.OutputBuffer.realWriteBytes(OutputBuffer.java:407)
> at: org.apache.tomcat.util.buf.ByteChunk.flushBuffer(ByteChunk.java:417)
> at: org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:339)
> at: org.apache.coyote.tomcat5.OutputBuffer.writeBytes(OutputBuffer.java:437)
> at: org.apache.coyote.tomcat5.OutputBuffer.write(OutputBuffer.java:424)
> at: org.apache.coyote.tomcat5.CoyoteOutputStream.write(CoyoteOutputStream.java:145)
> at: com.xxxxxxxxxxxxx.earth.GoogleEarthServlet.serveLightFile(GoogleEarthServlet.java:282)
> at: com.xxxxxxxxxxxxx.earth.GoogleEarthServlet.doGet(GoogleEarthServlet.java:166)
> at: javax.servlet.http.HttpServlet.service(HttpServlet.java:718)
> at: javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
> at: org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:411)
> at: org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:290)
> at: org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:271)
> at: org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:202)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at: com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
> at: org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:206)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at: org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
> at: org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
> at: org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:150)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
> at: org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
> at: org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
> at: org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
> at: org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:272)
> at: org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
> at: org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
> at: org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
> at: org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
> at: org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
> at: org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
> at: java.lang.Thread.run(Thread.java:619)
> [/code]
>
Hum...looks like Apache is not reading the bytes sent by mod_jk. That's
probably a configuration issue with Apache....now this is not something
I'm familiar with, unfortunately. If you increase the number of
connection between Apache and GF, would that help.
>> Are you able to get pages serviced when invoking
>> GF directly (let's say on port 8080)?
>
> Yes, that works:
That means the problem is not GlassFish, but the communication between
Apache via mod_jk. Let me dig a little into the code to see what I can
find (the mod_jk code).
Thanks
-- Jeanfrancois
>
> wget -O /dev/null http://server.domain:8080/Home.html
> --2008-08-28 09:23:01-- http://server.domain:8080/Home.html
> Auflösen des Hostnamen »server.domain«.... 127.0.0.1
> Verbindungsaufbau zu server.domain|127.0.0.1|:8080... verbunden.
> HTTP Anforderung gesendet, warte auf Antwort... 200 OK
> Länge: nicht spezifiziert [text/html]
> In »/dev/null« speichern.
>
>
> Of course we can send you the complete stacktrace and netstat-output via email.
>
>
> Thank you and with best regards
>
> Nitram
> [Message sent by forum member 'mleuschner' (mleuschner)]
>
> http://forums.java.net/jive/thread.jspa?messageID=295825
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>