users@grizzly.java.net

Re: Memory Usage in Grizzly on Glassfish 3.1

From: Eric Luong <eluong_at_ualberta.ca>
Date: Thu, 28 Jun 2012 10:39:04 -0600

Yes, all active at the same time.

On Thu, Jun 28, 2012 at 10:17 AM, Oleksiy Stashok <
oleksiy.stashok_at_oracle.com> wrote:

>
> On 06/28/2012 06:13 PM, Eric Luong wrote:
>
> OK, I will check it out while in the different states. However, I am not
> sure there is actually a leak. What I'm seeing is that memory usage seems
> to plateau after a certain number of clients, rather than continuing to
> increase linearly. I'm trying to understand why this happens.
>
> Just to make sure, all those clients (2000+) are active (in the
> long-polling state) at the same time, right?
>
> WBR,
> Alexey.
>
>
>
> Thanks,
>
> Eric
>
> On Thu, Jun 28, 2012 at 9:57 AM, Oleksiy Stashok <
> oleksiy.stashok_at_oracle.com> wrote:
>
>> Hi Eric,
>>
>> I think you're on the right way.
>> See inline...
>>
>>
>> On 06/28/2012 04:40 AM, Eric Luong wrote:
>>
>> Any suggestions on what to look for with jmap/jhat? I'm unfamiliar with
>> the tools. When I run jhat on a heap dump, it comes up with a fair number
>> of warnings -- "Failed to resolve object id [some id]" and "Failed to
>> resolve object id [some id] for field value (signature L)". Not sure what
>> these mean. It might be worth mentioning that I had to use -F on jmap;
>> without it, I get the "target process is not responding".
>>
>> That's correct.
>>
>>
>>
>> Just looking at this snapshot (which is not near the plateau I
>> described), it seems the class with the largest total size is "class [C"
>> which is obviously not a useful class name.
>>
>> It's character arrays.
>> I just took snapshot w/ jmap and opened in jhat.
>> In the browser I clicked "Heap histogram", now I see types sorted by
>> number of instances and memory consumption.
>>
>> When you investigate the problem, pls. take a snapshot when issue occurs
>> and check the heap histogram. Most probably the problematic object (which
>> leaks) would be on the top of the list.
>>
>>
>>
>> Does that mean it is some anonymous class? Exploring it further, it
>> only lists Object as its superclass. There are several other such nameless
>> classes. Other information for these classes (ClassLoader, Signers,
>> Protection Domain) are all null. Other classes that seem to consume the
>> majority of memory are HashMap, HashMapEntry and related classes.
>>
>> Most of the time char[] is at the top of the list, different collections
>> like HashMap also.
>> Pls. take a snapshot when the memory issue occurs and check the heap
>> histogram, may be you'll find some suspicious object on the top. If not -
>> you can share the snapshot file, I can try to help.
>>
>>
>>
>> Can you suggest how to approach this using jmap/jhat, and if those
>> warnings I received are of any significance?
>>
>> I also see warnings from jhat, but IMO it's still ok.
>>
>> You can also check another profiler tools like the one Chris suggested
>> (visual vm).
>>
>> Thanks.
>>
>> WBR,
>> Alexey.
>>
>>
>> Then I will proceed with checking out different memory state snapshots.
>>
>> Thanks for your help,
>>
>> Eric
>>
>> On Mon, Jun 25, 2012 at 6:37 AM, Oleksiy Stashok <
>> oleksiy.stashok_at_oracle.com> wrote:
>>
>>> Hi Eric,
>>>
>>> it would be great if you can monitor the memory usage using some
>>> profiler tool or jmap/jhat and give us more details on objects (object
>>> types), which consume memory. You can take several memory state snapshots
>>> like initial, 1000 clients connected, 2000 clients connected, 3000 clients
>>> connected, so we can see the dynamics and figure out the problem (if there
>>> is any).
>>>
>>> Thanks.
>>>
>>> WBR,
>>> Alexey.
>>>
>>>
>>> On 06/23/2012 04:48 AM, Eric Luong wrote:
>>>
>>>> Hello,
>>>>
>>>> I've been testing Grizzly in Glassfish 3.1 by simulating a load of
>>>> thousands of clients using long polling. I found what appears to be a
>>>> bottleneck. As more clients are added, the memory usage of the server
>>>> increases. However, after a certain point (2000 clients in my case, but I
>>>> expect it would depend on hardware) the usage seems to taper off and
>>>> plateau.
>>>>
>>>> I have been trying to figure out why the memory usage plateaus. Can
>>>> anyone offer some insights, or suggest a way to investigate further?
>>>>
>>>> Thank-you!
>>>>
>>>> Eric
>>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>