dev@jax-ws.java.net

Re: Major on-line production issue

From: LeRoy Hall <leroy_e_hall_at_yahoo.com>
Date: Thu, 12 Feb 2009 08:07:59 -0800 (PST)

Hey Jitu
 
I found out that the vendor servers connection timeout is set at 60 seconds.
 
We also did a load test last night and found that as long as there is a constant stream of messages then the same connections are re-used (persistent), but when the connection goes idle it is closed.  Is there anyway to configure this so that the connections remain open even when they are idle?
 
Thanks,
 
LeRoy

--- On Wed, 2/11/09, LeRoy Hall <leroy_e_hall_at_yahoo.com> wrote:

From: LeRoy Hall <leroy_e_hall_at_yahoo.com>
Subject: Re: Major on-line production issue
To: dev_at_jax-ws.dev.java.net
Date: Wednesday, February 11, 2009, 2:21 PM







Durring my test on a dev server they were not sending Connection: close.  In production the traffic is encrypted (SSL) so we can't see the headers.
 
In reference to connection timeouts, are you referring to the server we are connecting to or the server we are sending from?
 
The server we are sending from is running the JAX-WS clients as stand alone java applications.  They do not run under an application server, such as Tomcat.
 
I will ask our vendors what the connection timeout value is on their servers.
 
Thanks again

--- On Wed, 2/11/09, Jitendra Kotamraju <Jitendra.Kotamraju_at_Sun.COM> wrote:

From: Jitendra Kotamraju <Jitendra.Kotamraju_at_Sun.COM>
Subject: Re: Major on-line production issue
To: dev_at_jax-ws.dev.java.net
Date: Wednesday, February 11, 2009, 2:08 PM

LeRoy Hall wrote:
> They are returning HTTP 1.1 in the HTTP header, so we are assuming they
do. I have asked them to verify that their servers support HTTP 1.1. Should
know something later today.
> During the network sniff that we did we noticed that it was our server
(running the JAX-WS client) that was closing the connection.
>
Is the server sending Connection: close headers ?
> We also noticed that a few of the sockets were remaining open for a long
period of time. We believe this is because we are using the default
maxConnections of 5 when we have need for 30 or more per server. We are going
to set this to 100 and see if that has any effect. Do I use System.setProperty
to set this property or is there some JAX-WS specific way of doing it?
System./setProperty/("http.maxConnections", "100");
>
that should work. Or use java -Dhttp.maxConnections=100
> Our server engineers checked NIC cards on the servers and they are no
where near being overloaded. CPU utilization during peak message volume never
exceeded 25%, and we had plenty of free memory. I did increase the connection
timeout from 1 second to 2 seconds, but this
>
Are you increasing the connection timeout on the server(tomcat or GlassFish or
...) ?

thanks,
Jitu
> didn't work either. 1 second is an eternity to establish a socket
connection, wouldn't you think?
> I will be interested to learn what you find out from the JDK team.
> Thanks Jitu.
>
>
> --- On *Wed, 2/11/09, Jitendra Kotamraju
/<Jitendra.Kotamraju_at_Sun.COM>/* wrote:
>
> From: Jitendra Kotamraju <Jitendra.Kotamraju_at_Sun.COM>
> Subject: Re: Major on-line production issue
> To: dev_at_jax-ws.dev.java.net
> Date: Wednesday, February 11, 2009, 11:35 AM
>
> LeRoy Hall wrote:
> > I tried that this morning. The behavior is the same as 2.1.3.
The
> connection is opened and reused, but if it is idle for more than a
couple of
> seconds it gets dropped. It doesn't remain persistent.
> > Probably, that's the behaviour of JDK's
HttpURLConnection. I will check
> with someone from JDK team. Do you know the vendor's sever
supports
> keep-alive ?
>
> > Do you believe that this new version of JAX-WS will eliminate
the
> connection exceptions we are seeing? If so, can you explain why or
point out
> some bug fix since 2.1.3 that has fixed this issue?
> > 2.1.5 may have fixed some keep-alive issues in some corner
cases(esp reading
> the HttpURLConnection's OutputStream). We follow
>
http://java.sun.com/javase/6/docs/technotes/guides/net/http-keepalive.html
>
> Regarding the timeouts, may be the server is overloaded. or the server
may have
> to increase the connection time(For example, few users modified the
connection
> settings on tomcat and that fixed the timeout issues)
>
> Jitu
> > > > --- On *Tue, 2/10/09, Rama Pulavarthi
/<Rama.Pulavarthi_at_Sun.COM>/*
> wrote:
> > > From: Rama Pulavarthi
<Rama.Pulavarthi_at_Sun.COM>
> > Subject: Re: Major on-line production issue
> > To: dev_at_jax-ws.dev.java.net
> > Date: Tuesday, February 10, 2009, 11:22 PM
> > > Hi LeRoy,
> > Did you try with latest JAX-WS RI 2.1.5?
> > > thanks,
> > Rama Pulavarthi
> > > LeRoy Hall wrote:
> > > Hello all,
> > > I've got a major production issue that I'm
desperately
> hoping
> > someone out there can help with. We use JAX-WS clients to
connect
> with our
> > vendors to send OTA (open travel alliance) XML messages.
These
> clients are
> > running as stand-alone java applications (no application
servers
> involved) We
> > have one vendor in particular that we are sending well over a
million
> messages a
> > day, and we recently started taking the following exceptions
on 1% of
> our
> > messages:
> > > com.sun.xml.ws.client.ClientTransportException: HTTP
transport
> error:
> > java.net.SocketTimeoutException: connect timed out
> > > We had the connection timeout set to 1 second, and I
increased
> it to 2
> > seconds which had no effect. We also had network engineers
on both
> sides, and
> > in between (Verizon) looking into this and found nothing from
a
> network point of
> > view that would be causing this.
> > > Our server engineers checked out the servers NIC cards
and no
> problems
> > there. I even loaded the clients onto two new servers and it
still
> continues to
> > take these exceptions.
> > > We did discover that the socket connections are not
remaining
> persistent.
> > JAX-WS is opening and closing a connection for every
message. We are
> re-using
> > the JAX-WS proxy objects (port type objects) returned from
the JAX-WS
> service
> > objects generated by wsimport, so I am unsure why it is
behaving this
> way, and
> > am not finding much help concerning this issue on the
internet.
> > > We are using JAX-WS RI 2.1.3-b02- running on windows
servers. CPU
> > utilization is very low (never higher than 25%) and memory is
fine. Any help
> > you could provide would be fantastic.
> > > Thanks,
> > > LeRoy
> > > > > >
---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe_at_jax-ws.dev.java.net
> > For additional commands, e-mail: dev-help_at_jax-ws.dev.java.net
> > > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_jax-ws.dev.java.net
> For additional commands, e-mail: dev-help_at_jax-ws.dev.java.net
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe_at_jax-ws.dev.java.net
For additional commands, e-mail: dev-help_at_jax-ws.dev.java.net