Hi,
We have been developing a system on glassfish for some time with no problems.
However, we are now moving to production and have to proxy through apache.
The problem is that when you first start apache it forwards over ajp fine but after a little while apache starts returning 503s.
Errors seen in apache error_log:
[Mon Jun 23 16:23:18 2008] [error] (70007)The timeout specified has expired: ajp_ilink_receive() can't receive header
[Mon Jun 23 16:23:18 2008] [error] ajp_read_header: ajp_ilink_receive failed
[Mon Jun 23 16:23:18 2008] [error] (120006)APR does not understand this error code: proxy: read response failed from 172.22.8.195:5089 (172.22.8.195)
Errors seen in glassfish server.log:
[#|2008-06-23T16:31:20.893+0100|WARNING|sun-appserver9.1|org.apache.jk.core.MsgContext|_ThreadID=28;_ThreadName=TP-Processor24;_RequestID=d0a93dcb-7347-4702-8ea4-ff6147c08e79;|Error sending end packet
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
at sun.nio.ch.IOUtil.write(IOUtil.java:75)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.sun.enterprise.server.ss.provider.ASOutputStream.write(ASOutputStream.java:151)
at org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:537)
at org.apache.jk.common.JkInputStream.endMessage(JkInputStream.java:127)
at org.apache.jk.core.MsgContext.action(MsgContext.java:302)
at org.apache.coyote.Response.action(Response.java:221)
at org.apache.coyote.Response.finish(Response.java:346)
at org.apache.coyote.tomcat5.OutputBuffer.close(OutputBuffer.java:324)
at org.apache.coyote.tomcat5.CoyoteResponse.finishResponse(CoyoteResponse.java:578)
at org.apache.coyote.tomcat5.CoyoteAdapter.afterService(CoyoteAdapter.java:318)
at org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:301)
at org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773)
at org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703)
at org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:666)
at java.lang.Thread.run(Thread.java:619)
|#]
Sending a HUP to apache solves this problem for a while but it soon stops working again. I have tried several options to ProxyPass specifying timeouts etc but none seem to make any difference.
I followed Jean-Francois's and Amy Roh's blogs to set this up as follows:
Apache (recently upgraded to rule it out as a cause of the problems) accepting https and requiring mutual authentication:
# apache2ctl -version
Server version: Apache/2.2.8 (Linux/SUSE)
Server built: Jun 6 2008 23:42:07
mod_proxy, mod_proxy_ajp:
ProxyPass /client ajp://172.22.8.195:5089/client/
ProxyPassReverse /client ajp://172.22.8.195:5089/client/
Glassfish running on 172.22.8.195 listening for ajp and https requiring mutual authentication:
domain.xml:
<jvm-options>-Dcom.sun.enterprise.web.connector.enableJK.propertyFile=${com.sun.aas.instanceRoot}/config/glassfish-jk.properties</jvm-options>
<jvm-options>-Dcom.sun.enterprise.web.connector.enableJK=5089</jvm-options>
glassfish-jk.properties:
port=5089
# java -version
java version "1.6.0_05"
Java(TM) SE Runtime Environment (build 1.6.0_05-b13)
Java HotSpot(TM) 64-Bit Server VM (build 10.0-b19, mixed mode)
tomcat-ajp.jar from suse 10.3 64bit package tomcat 5.5
The application uses web services for communication, and the client specifies fast infoset.
I have googled for these errors but none of the other posts I have come across have helped.
Any ideas?
Thanks,
Will
[Message sent by forum member 'willhollins' (willhollins)]
http://forums.java.net/jive/thread.jspa?messageID=282032