dev@grizzly.java.net

Re: slow Admin Console load time

From: Anissa Lam <anissa.lam_at_oracle.com>
Date: Mon, 03 Jan 2011 08:30:17 -0800

Thanks Jason for looking into this. I think we should update the issue
with all the findings. Harder to keep track using email and there
maybe others not in the alias is interested in this also.

I will update the bug with your comments.

thanks
Anissa.

On 1/3/11 8:14 AM, Jason Lee wrote:
> I added logging statements at various points in admin console adapter
> and installer thread. Here's what I got:
>
> ***** 09:52:40.0944: Application is not yet loaded.
> ***** 09:52:40.0945: Force REST module load
> ***** 09:52:40.0947: Starting installer thread
> ***** 09:52:40.0954: InstallThread.install()
> ***** 09:52:40.0956: building ConfigCode
> ***** 09:52:40.0957: Getting server
> ***** 09:52:40.0957: Applying changes
> ***** 09:52:41.0668: Changes applied
> ***** 09:52:41.0671: Getting ApplicationRef
> ***** 09:52:41.0672: Process application
> ***** 09:52:42.0346: Application is not yet loaded.
> ***** 09:52:59.0675: Application processed
>
> The 'Process application' and 'Application processed' entries show the
> greatest difference. Those messages came from these lines of code:
>
> adapter.logTime("Process application");
>
> habitat.getComponent(ApplicationLoaderService.class).processApplication(config,
> ref, log);
> adapter.logTime("Application processed");
>
> That one line of code took 18 seconds. There may be other hot spots
> (I'm still looking and tracing), but that seems like a good one to
> look at.
>
>
> On 1/3/11 3:40 AM, Oleksiy Stashok wrote:
>> Such a long delays don't look like Grizzly perf. issues.
>>
>> Alexey.
>>
>> On Dec 30, 2010, at 23:29 , shaline gowda wrote:
>>
>>> I Agree with Harshad, Slow load time for GUI with DAS single
>>> instance is been there all the time, and I always have to clear
>>> cache and do shift +reload to refresh the screen.
>>>
>>> -Shaline.
>>>
>>> Harshad Vilekar wrote:
>>>> Slow load time / GUI blank screen was seen earlier in build 32.
>>>> That was for DAS single instance - with no secure admin. It took
>>>> more than 40 seconds to load. Our main focus was functional
>>>> testing - so we moved on with workarounds like pressing shift
>>>> reload (for blank screen).
>>>>
>>>> Harshad.
>>>>
>>>> On 12/30/10 13:36, Ludovic Champenois wrote:
>>>>> On Dec 30, 2010, at 1:31 PM, Sudipa Bhattacharya<sudipa.bhattacharya_at_oracle.com> wrote:
>>>>>
>>>>>
>>>>>> Harshad, Please file a bug as Nazrul suggested and mark as Blocking .
>>>>>>
>>>>>> Lidia also had few observations , where she saw 4-5 minutes response time for simple operations in cluster.
>>>>>> She also had an observation that more the server is running (1 or 2 days), slower it gets .
>>>>>>
>>>>>
>>>>> Gui or cli?
>>>>>
>>>>> Is it new or regression or never tested before?
>>>>>
>>>>>
>>>>>> Sometimes, the page is not even loaded , the screen goes white after 4-5 minutes.
>>>>>>
>>>>>> She could not file a bug as she needed to confirm her findings further, I will ask her to update this bug as she comes back from vacation next week.
>>>>>>
>>>>>> Thanks all for your attention to this issue !
>>>>>>
>>>>>> --Sudipa
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 12/30/10 1:24 PM, Tim Quinn wrote:
>>>>>>
>>>>>>> Adding the Grizzly folks.
>>>>>>>
>>>>>>> I know that Ryan found and fixed a performance issue in Grizzly with SSL some time ago. Is there any chance that it has crept back in somehow or some other change is slowing down SSL traffic?
>>>>>>>
>>>>>>> - Tim
>>>>>>>
>>>>>>>
>>>>>>> On Dec 30, 2010, at 2:45 PM, Nazrul Islam wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Hi Tim/GUI team/Tom,
>>>>>>>>
>>>>>>>> Thanks. Lets take a look at this.
>>>>>>>>
>>>>>>>> 200+ sec (3+ min) is not acceptable response time. This is a show stopper issue for 3.1.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Nazrul Islam - (408) 276-6468 - Oracle
>>>>>>>>
>>>>>>>>
>>>>>>>> Tim Quinn wrote:
>>>>>>>>
>>>>>>>>> On Dec 30, 2010, at 2:34 PM, Nazrul Islam wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi Tim,
>>>>>>>>>>
>>>>>>>>>> For use-case #3, Harshad is reporting 200+ sec just to load admin console application. We will need to understand why presence of cluster/server instance (with secure-admin ?) increases the overall load time; i.e. time user needs to wait until they see the initial console page.
>>>>>>>>>>
>>>>>>>>> There were problems in Grizzly in earlier builds that have been fixed - long before b36.
>>>>>>>>>
>>>>>>>>> Obviously the first step is to understand exactly where the time is being spent.
>>>>>>>>>
>>>>>>>>> - Tim
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I filed the following issue earlier for admin console loading in DAS (use-case #1):http://java.net/jira/browse/GLASSFISH-14451
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Nazrul Islam - (408) 276-6468 - Oracle
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Tim Quinn wrote:
>>>>>>>>>>
>>>>>>>>>>> On Dec 30, 2010, at 1:59 PM, Nazrul Islam wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> [Adding Tim, Tom, Ludo to the thread]
>>>>>>>>>>>>
>>>>>>>>>>>> Hi Harshad,
>>>>>>>>>>>>
>>>>>>>>>>>> Please file a bug.
>>>>>>>>>>>>
>>>>>>>>>>>> Tom/Tim: Have we looked at DAS and cluster startup time after admin security was added?
>>>>>>>>>>>>
>>>>>>>>>>> I did not do much performance analysis, but some informal runs I did showed very little difference. I had created a cluster with two instances, with both instances and the DAS on my development system.
>>>>>>>>>>>
>>>>>>>>>>> With secure admin disabled, starting the domain and the cluster took 1m 22s elapsed. With secure admin enabled it took 1m 24s elapsed.
>>>>>>>>>>>
>>>>>>>>>>> I did not run the tests repeatedly to take averages, but I did measure the second start-domain + start-cluster run in each case so the first one would do equivalent 'warm-up" and OS caching, etc.
>>>>>>>>>>>
>>>>>>>>>>> FWIW,
>>>>>>>>>>>
>>>>>>>>>>> - Tim
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Nazrul Islam - (408) 276-6468 - Oracle
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Harshad Vilekar wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Anissa,
>>>>>>>>>>>>>
>>>>>>>>>>>>> We've been seeing slow load time for Admin Console since quite some time.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here are my observations with latest nightly build (ogs-3.1-b36-12_30_2010.zip). It shows the time it takes for Admin Console screen to show up in the GUI, after entering the URL in the browser, and pressing return. In case of https, the time is counted after confirming security exception prompt. This is by no means any methodical performance test, but simply shows "perceived load time" on the Solaris 10 Sparc machines (Niagara, T1000), that I use for day to day testing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1. DAS only (secure admin enabled): 30+ sec
>>>>>>>>>>>>> https://qm2.us.oracle.com:4848/common/index.jsf
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2. Single node, two instance cluster (secure admin not enabled): 100+ sec
>>>>>>>>>>>>> http://intg2t1000.us.oracle.com:4848/common/index.jsf
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3. Two node, two instance cluster (secure admin enabled): 200+ sec
>>>>>>>>>>>>> https://intg3t1000.us.oracle.com:4848/common/index.jsf
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we need to address this, especially case #3. I would appreciate your feedback on this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Harshad.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>
>>>
>>
>
>
> --
> Jason Lee
> Senior Member of Technical Staff
> GlassFish REST API / Administration Console
>
> Oracle Corporation
> Phone +1 405-216-3193
> Bloghttp://blogs.steeplesoft.com