On Jun 8, 2010, at 5:21 PM, Alex Sherwin wrote:
> Poking about the source a bit, and it looks like all you would need
> to do is set the property on the ClientRequest object [ i.e.
> clientrequest
> .getProperties().put(ClientConfig.PROPERTY_READ_TIMEOUT, 1000) ]
>
> To me, it looks like the easiest place to do this (as a one-off
> solution) would be in WebResource; but the area you would want to
> modify would be the build() methods or the handle() methods... but
> both of which are private, so you'd have to c/p the whole class and
> supply your own copy on the applications classpath before the jersey
> libs.
>
> I could easily be missing an easier place to do it, but I just
> noticed this possibility
>
The problem is the handler for the WebResource is called before the
handler for the Client, which currently does the following:
public ClientResponse handle(final ClientRequest request) throws
ClientHandlerException {
request.getProperties().putAll(properties);
request.getProperties().put(Client.class.getName(), this);
final ClientResponse response =
getHeadHandler().handle(request);
response.getProperties().put(Client.class.getName(), this);
return response;
}
Notice that it copies all properties from the Client to the
ClientRequest, overwriting any existing properties so we cannot
currently guarantee that the correct property value will be set.
One can set the property using a filter added to WebResource, there is
no need to override WebResource. If you can guarantee that the
property is never set on the client then this will work.
A simple fix would be to only set properties if not already present.
However, a better fix is to support properties on WebResource and have
a copy on write mechanism so that we can avoid lots of copying for
common cases.
Paul.
> On 6/8/2010 10:56 AM, Mike Boyers wrote:
>> Paul,
>>
>> Thanks for the response. I looked at the source for
>> ApacheHttpClientHandler and I totally see what's going on now.
>>
>> Is a request-specific timeout something you would consider for a
>> future release?
>>
>> Do you have an idea about how the change would be implemented? Do
>> you envision it being a relatively simple change? Is it something
>> a newcomer might be able to contribute, or at least help with?
>>
>> Thanks,
>> Mike
>>
>> --- On Tue, 6/8/10, Paul Sandoz<Paul.Sandoz_at_Sun.COM> wrote:
>>
>>
>>> From: Paul Sandoz<Paul.Sandoz_at_Sun.COM>
>>> Subject: Re: [Jersey] jersey-apache-client connection pool
>>> configuration
>>> To: users_at_jersey.dev.java.net
>>> Date: Tuesday, June 8, 2010, 1:57 AM
>>> Hi Mike,
>>>
>>> As you are doing you can set the read timeout on the Client
>>> properties:
>>> https://jersey.dev.java.net/nonav/apidocs/latest/jersey/com/sun/jersey/api/client/Client.html
>>> #setReadTimeout%28java.lang.Integer%29
>>>
>>> The Apache HTTP client will take this value and set it on
>>> the HttpMethodParams of the HttpMethod.
>>>
>>> // Set the read timeout
>>> final Integer readTimeout =
>>> (Integer)props.get(ApacheHttpClientConfig.PROPERTY_READ_TIMEOUT);
>>> if (readTimeout != null) {
>>> methodParams.setSoTimeout(readTimeout);
>>> }
>>>
>>> So it is up to the Apache HTTP client as to how it manages
>>> that w.r.t. multiple threads etc.
>>>
>>> This property is not currently one that you can set
>>> per-request or per-WebResource or when building a request.
>>> Currently it is not possible to override with properties on
>>> a WebResource, it is something we have looked into but never
>>> got around to fully implementing it.
>>>
>>> The only work around i can think of at the moment is to
>>> create two Client instances with separate read time out
>>> values. Not particular efficient.
>>>
>>> Paul.
>>>
>>> On Jun 8, 2010, at 12:01 AM, Mike Boyers wrote:
>>>
>>>
>>>> Alex,
>>>>
>>>> Thanks - your response has been very helpful.
>>>>
>>>> Yes, configuring the poolsize in the way you described
>>>>
>>> will work for me.
>>>
>>>> I can now also envision how to set the connection
>>>>
>>> manager timeout and I also think I can use a
>>> ClientFilter to record all of the JMX stats I need
>>> (timeouts, pool timeouts, total requests, durations, etc.)
>>>
>>>> However, I'm now hung up on request-specific
>>>>
>>> timeouts.
>>>
>>>> My understanding is that I should declare the Client,
>>>>
>>> then allow it to be shared across all threads.
>>>
>>>> The only way I've found to set the request timeout is
>>>>
>>> on the shared Client itself, via the setReadTimeout()
>>> method.
>>>
>>>> So, I can do something similar to what you typed
>>>>
>>> initially:
>>>
>>>> httpClient.setReadTimeout(2000);
>>>> ClientResponse response =
>>>>
>>> httpClient.resource("/some/
>>> uri").accept(..).get(ClientResponse.class);
>>>
>>>> if (response.getStatus()<= 201) {
>>>> String entity =
>>>>
>>> response.getEntity(String.class);
>>>
>>>> }
>>>>
>>>> My concern is one thread trampling on another's
>>>>
>>> timeout.
>>>
>>>> I plan on making multiple requests concurrently via a
>>>>
>>> java.util.ThreadPoolExecutor's invokeAll() method. So
>>> I set up a test with an executor where I'm sending 2
>>> requests concurrently to the same resource. One of
>>> them has a timeout of 1000ms, the other 9000ms, and I've
>>> configured the remote resource to not respond with anything
>>> until 10 seconds, so both of my client requests should time
>>> out.
>>>
>>>> In my testing, sometimes the behavior is correct, one
>>>>
>>> request will timeout after 1000ms and the other after
>>> 9000ms. But most often, both requests end up with the
>>> same timeout, whether it be 9000ms or 1000ms, and that's not
>>> good in my case, since this test is actually pretty close to
>>> what I'll need to do in production.
>>>
>>>> Is there a "request-specific" way to set a timeout
>>>>
>>> that I've overlooked?
>>>
>>>> Thanks,
>>>> Mike
>>>>
>>>> --- On Mon, 6/7/10, Alex Sherwin<alex.sherwin_at_acadiasoft.com>
>>>>
>>> wrote:
>>>
>>>>
>>>>> From: Alex Sherwin<alex.sherwin_at_acadiasoft.com>
>>>>> Subject: Re: [Jersey] jersey-apache-client
>>>>>
>>> connection pool configuration
>>>
>>>>> To: users_at_jersey.dev.java.net
>>>>> Date: Monday, June 7, 2010, 12:48 PM
>>>>> How are you configuring your
>>>>> client? I'm using the ApacheHttpClient
>>>>>
>>> wrapper from
>>>
>>>>> the Jersey libs, and setting up the
>>>>> MultiTHreadedHttpConnectionManager directly, like
>>>>>
>>> so:
>>>
>>>>> connectionManager =
>>>>>
>>> new
>>>
>>>>> MultiThreadedHttpConnectionManager();
>>>>>
>>>>>
>>>>>
>>> connectionManager
>>> .getParams
>>> ().setDefaultMaxConnectionsPerHost(MAX_CONNECTIONS_PER_HOST);
>>>
>>>>>
>>>>>
>>> connectionManager
>>> .getParams().setMaxTotalConnections(MAX_TOTAL_CONNECTIONS);
>>>
>>>>> clientConfig = new
>>>>> DefaultApacheHttpClientConfig();
>>>>>
>>>>>
>>>>>
>>> clientConfig
>>> .getProperties
>>> ().put
>>> (DefaultApacheHttpClientConfig.PROPERTY_PREEMPTIVE_AUTHENTICATION,
>>>
>>>>> Boolean.TRUE);
>>>>>
>>>>> httpClient = new
>>>>>
>>> ApacheHttpClient(new
>>>
>>>>> ApacheHttpClientHandler(new
>>>>>
>>> HttpClient(connectionManager)),
>>>
>>>>> clientConfig);
>>>>>
>>>>> It sounds like this is what you want to do, I
>>>>>
>>> believe
>>>
>>>>> Then, to use,
>>>>>
>>>>> ClientResponse response =
>>>>>
>>>>>
>>> httpClient.resource("/some/
>>> uri").accept(..).get(ClientResponse.class);
>>>
>>>>> if (response.getStatus()<= 201) {
>>>>> String entity =
>>>>>
>>> response.getEntity(String.class);
>>>
>>>>> }
>>>>>
>>>>> Etc
>>>>>
>>>>>
>>>>> On 6/7/2010 12:08 PM, Mike Boyers wrote:
>>>>>
>>>>>> I'm beginning work on a RESTful api that, in
>>>>>>
>>> order to
>>>
>>>>> fulfill requests, will need to reach out to other
>>>>>
>>> apis (some
>>>
>>>>> RESTful themselves, some HTTP but not RESTful).
>>>>>
>>>>>> My main requirements for any HTTP requests
>>>>>>
>>> (both
>>>
>>>>> RESTful and non-RESTful) made to other services
>>>>>
>>> are:
>>>
>>>>>> -Support keep-alives
>>>>>> -Support connection pooling
>>>>>> -Support maximum configurable sizes for
>>>>>>
>>> connection
>>>
>>>>> pools
>>>>>
>>>>>> -Support fail-fast "connection pool timeouts"
>>>>>>
>>> when a
>>>
>>>>> particular pool has reached its maximum configured
>>>>>
>>> size
>>>
>>>>>> -If a pool grows under load, it needs to
>>>>>>
>>> shrink again
>>>
>>>>> as load decreases
>>>>>
>>>>>> -Both RESTful and non-RESTful request should
>>>>>>
>>> share the
>>>
>>>>> same connection pools
>>>>>
>>>>>> -Ability to query for pool sizes and other
>>>>>>
>>> statistics
>>>
>>>>> to expose via JMX
>>>>>
>>>>>> When using Apache's HttpClient in the raw, I
>>>>>>
>>> know I
>>>
>>>>> can accomplish all of these goals. But that
>>>>>
>>> won't get
>>>
>>>>> me the jersey-client features I'd like to have, so
>>>>>
>>> I'd like
>>>
>>>>> to consider using the jersey-apache-client for all
>>>>>
>>> HTTP
>>>
>>>>> requests and just get the non-RESTful responses as
>>>>>
>>> Strings.
>>>
>>>>>> However, I'm not sure how I can configure the
>>>>>>
>>>>> underlying MultiThreadedHttpConnectionManager to
>>>>>
>>> behave the
>>>
>>>>> way I want. Some of the methods I'm used to
>>>>>
>>> calling
>>>
>>>>> are:
>>>>>
>>>>>> setDefaultMaxConnectionsPerHost()
>>>>>> setConnectionTimeout() -- I know this is
>>>>>>
>>> already
>>>
>>>>> supported and know how to configure it
>>>>>
>>>>>> closeIdleConnections()
>>>>>> deleteConnections()
>>>>>>
>>>>>> And also HttpClientParameter's
>>>>>>
>>>>> setConnectionManagerTimeout() method.
>>>>>
>>>>>> Any ideas?
>>>>>>
>>>>>> Thanks,
>>>>>> Mike Boyers
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>>>>>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>> ---------------------------------------------------------------------
>>>
>>>>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>>>>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>>
>>>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>>>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>>
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
>> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
> For additional commands, e-mail: users-help_at_jersey.dev.java.net
>