On 08/24/2010 12:30 PM, Rama Pulavarthi wrote:
> Hi,
> Most of it might be taken by Jaxb for the creation of JAXBContext.
>>> The app in question is a daemon that downloads data from many
> different Salesforce organizations using the Partner WSDL.
> Do all the organizations implement the same wsdl contract? Are you
> creating a separate client instance to talk to each organization?
Yes, and yes. Actually, I create many clients per org so that multiple
threads can download data simultaneously.
One thing I was considering was to only create a few clients that would
be shared by every org by reconfiguring the soap headers and endpoint
every time they were used.
>
>>> It looks like the binding is holding on to too much data about the
> last request that was performed on it. (If you doubt the idle-ness of
> the app, this particular connection object was in the connection pool,
> so you may be doubly confident that this one was not in use.)
>
> Yes, the ResponseContext has the information about last invocation and
> it it still lying around as we don't know when we can throw away that
> information as the user can query the response context anytime after
> invocation. Unless you keep the references to these proxies, it should
> have been all garbage collected. Seems like you are keeping the proxy
> for reuse. Currently, we don't support
> BindingProvider.getResponseContext().clear() as its implemented as read
> only. We could use this api to make the runtime forget about the last
> invocation. Please file an issue at
> https://jax-ws.dev.java.net/servlets/ProjectIssues
I do keep the references around. Should I file it as an "Enhancement"?
>
> Based on your usage, you can also try reusing the JAXBContext, if it is
> shared by multiple services. See
> http://weblogs.java.net/blog/jitu/archive/2008/08/control_of_jaxb.html.
> I think, if you reuse the JAXBContext across the different proxies you
> use for each organization, it should help.
>
> thanks,
> Rama Pulavarthi
Could you be more specific about how to use this on a client? I think I
see how to use it on a web service whose source I control, but I'm
generating my source with wsimport. How can I set the generated source
to use one JAXBContext? I'm not doing anything special with JAXB aside
from a customization file for wsimport to avoid class name clashes from
the wsdl, so as long as JAXBContext is thread safe I suppose it should
be fine for everything to share one instance.
Thanks,
Marshall
>
> On 08/23/2010 03:05 PM, Marshall Pierce wrote:
>> I'm seeing pretty high memory usage with idle JAX-WS client bindings
>> using 2.1.7. (I haven't upgraded to 2.2 because of the need to deploy
>> jars into the jvm endorsed dir.)
>>
>> The app in question is a daemon that downloads data from many
>> different Salesforce organizations using the Partner WSDL. (Salesforce
>> is a multi-tenant system, so if you were to buy their service you
>> would be one of many orgs that reside on the same server cluster.)
>> Because our systems talk to many different orgs, we cache several
>> bindings for each org.
>>
>> I have a heap dump from when the app was idle. (I abused the GC button
>> in jconsole until memory usage stabilized just before I took the heap
>> dump, and I'm pretty sure I only included live objects in the heap
>> dump anyway.) There were 3,056 cached bindings and the heap dump is
>> 5.3GB.
>>
>> jhat's basic data about the heap dump is pretty telling. The first
>> several dozen classes in the instance count view are all
>> com.sun.xml.bind.v2. Here's the top of the instance count view:
>>
>> 2060410 com.sun.xml.bind.v2.runtime.Name
>> 1825554 com.sun.istack.FinalArrayList
>> 1782624 com.sun.xml.bind.v2.util.QNameMap$Entry
>> 1489196 com.sun.xml.bind.v2.runtime.unmarshaller.ChildLoader
>> 1344078 com.sun.xml.bind.v2.model.impl.FieldPropertySeed
>> 1344078
>> com.sun.xml.bind.v2.model.impl.RuntimeClassInfoImpl$RuntimePropertySeed
>> 1344078 com.sun.xml.bind.v2.runtime.reflect.Accessor$FieldReflection
>> 1325125 com.sun.xml.bind.v2.model.impl.ElementPropertyInfoImpl$1
>> 1325125 com.sun.xml.bind.v2.model.impl.RuntimeElementPropertyInfoImpl
>> 1325125 com.sun.xml.bind.v2.model.impl.RuntimeTypeRefImpl
>> 1113715 com.sun.xml.bind.v2.runtime.output.Encoded
>> 883400 com.sun.xml.bind.v2.model.impl.RuntimeEnumConstantImpl
>> 760363 com.sun.xml.bind.v2.runtime.unmarshaller.LeafPropertyLoader
>> 725667 com.sun.xml.bind.v2.runtime.property.TagAndType
>>
>> 2 million Names seems a little excessive for a completely idle app,
>> even though at 39 bytes each that doesn't add up to all *that* much
>> memory.
>>
>> One interesting thing is that the idle memory usage increases as we
>> increase the amount of data we get per call from Salesforce. One of
>> the calls (retrieve()) lets you specify the list of Ids that you want
>> the data for with a max of 2000 Ids. We want to use as many Ids as
>> possible in each call to minimize the number of API calls made since
>> those are metered by Salesforce. We used to use 500 per call, but
>> recently increased it to use fewer API calls. When we tried to
>> download 2000 per retrieve(), we quickly got an OOME. We're currently
>> using 1000 per call. That OOME (always fun on production...) is what
>> prompted me to start looking into JAX-WS's memory usage.
>>
>> No doubt you guys will have further suggestions for tracking down the
>> source of the issue, but I did find one interesting thing just poking
>> around in jhat. The highest instance count for any clas in our package
>> (com.genius) was for a class that JAX-WS generated from the Partner
>> WSDL. It happened to be QueryAll which is used to house the SOQL (it's
>> like SQL but for Salesforce objects) statement passed to the queryAll
>> API call. The interesting part is the reference chain from the rootset
>> to a particular instance (chosen randomly -- other instances had
>> similar or identical chains).
>>
>> --> com.genius.crm.sf.conn.ConnectionImpl_at_0x7f06ba6f9400 (81 bytes)
>> (field binding:)
>> --> $Proxy50_at_0x7f06ba6f95f8 (24 bytes) (field h:)
>> --> com.sun.xml.ws.client.sei.SEIStub_at_0x7f06ba6f9610 (128 bytes)
>> (field responseContext:)
>> --> com.sun.xml.ws.client.ResponseContext_at_0x7f07478cd2c8 (48 bytes)
>> (field packet:)
>> --> com.sun.xml.ws.api.message.Packet_at_0x7f07478cd2f8 (145 bytes)
>> (field satellites:)
>> --> com.sun.istack.FinalArrayList_at_0x7f07478cd390 (32 bytes) (field
>> elementData:)
>> --> [Ljava.lang.Object;@0x7f07478cd3b8 (96 bytes) (Element 1 of
>> [Ljava.lang.Object;@0x7f07478cd3b8:)
>> -->
>> com.sun.xml.ws.transport.http.client.HttpResponseProperties_at_0x7f07478cd420
>> (32 bytes) (field deferedCon:)
>> -->
>> com.sun.xml.ws.transport.http.client.HttpClientTransport_at_0x7f07478cd440 (93
>> bytes) (field context:)
>> --> com.sun.xml.ws.api.message.Packet_at_0x7f07478da2c8 (145 bytes)
>> (field message:)
>> --> com.sun.xml.ws.message.jaxb.JAXBMessage_at_0x7f07478da3f0 (112 bytes)
>> (field jaxbObject:)
>> --> com.genius.crm.sf.jaxwsstub.QueryAll_at_0x7f07478da528 (24 bytes)
>>
>> com.genius.crm.sf.conn.ConnectionImpl is my own wrapper around the
>> JAX-WS-generated binding (the $Proxy50 object).
>>
>> It looks like the binding is holding on to too much data about the
>> last request that was performed on it. (If you doubt the idle-ness of
>> the app, this particular connection object was in the connection pool,
>> so you may be doubly confident that this one was not in use.)
>>
>> I'm happy to enable extra logging, etc to help track this down. Memory
>> usage is currently the #1 bottleneck facing this app.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>