dev@glassfish.java.net

Re: QL failures with current GF 3.1 build

From: Marina Vatkina <marina.vatkina_at_oracle.com>
Date: Tue, 17 Aug 2010 15:03:53 -0700

I got one more problem with the clusterHelloWorldTest:

   [testng] problem initializing - cycle detected involving:
com.sun.hk2.component.SingletonInhabitant_at_52df636. Please see server.log
for more details.

When run separately ('cluster/clustersetup' followed by
'cluster/helloworld') all tests pass.

-marina

Tim Quinn wrote:
> Tom and I exchanged a few e-mails off-line.
>
> The admin command requests don't flow through the web container.
>
> It seems plausible that there is some start-up issue that affects both
> the AdminAdapter (which is why admin commands think the server is not
> up) and the web container.
>
> Is it possible that Grizzly adapters are started up asynchronously?
> If so, then that could explain why an instance claims to have started
> but an immediate asadmin command or web probe fails.
>
> - Tim
>
> On Aug 17, 2010, at 9:34 AM, Tom Mueller wrote:
>
>> 13002 is a duplicate of 12967. It's easy to reproduce this problem by
>> just doing the following:
>> asadmin start-local-instance --node localhost ix; wget -S --no-proxy http://localhost:28080/ <http://localhost:28084/hellox/hi.jsp>
>> This will show the web container returning a 500 error code
>> immediately after the instance is started. This is being investigated.
>> The reason that list-instances fails is because it calls __asadmin
>> twice on the instance, so it is seeing the same problem.
>>
>> Tom
>>
>>
>> On 8/17/2010 8:08 AM, Jane Young wrote:
>>> Tim Quinn wrote:
>>>> Hi, Jane.
>>>>
>>>> On Aug 17, 2010, at 7:29 AM, Jane Young wrote:
>>>>
>>>>> Hi Tim,
>>>>>
>>>>> It's failing for me when running QL cluster_setup test by itself.
>>>>> I think this is a performance regression. With the latest
>>>>> changes, the startup time is taking longer. QL is doing
>>>>> "list-instances" right after the "start-instance" command to check
>>>>> if the instance is up and running. Since it's taking a longer
>>>>> time for instance to startup, list-instances is returning false.
>>>>> I added extra 5 sec wait in QL after start-instance and the tests
>>>>> are passing. Same goes for cluster_helloworld tests. I'll commit
>>>>> the changes in QL tests.
>>>>
>>>> If start-instance reports success before the instance is really
>>>> ready to do work, then that's a bug. Inserting the 5-second pause
>>>> makes sense to get the QL tests passing again but I hope someone
>>>> will investigate why the instance claims to be up when it is not
>>>> really up.
>>> Agreed that this is a workaround in QL. Bug filed for the
>>> start-instance command:
>>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=13002
>>> What about the cluster_helloworld test - is that also a bug? QL is
>>> trying to access the helloworld app right after deployment and it's
>>> taking a longer time to generate an app.
>>>
>>>
>>>>
>>>> - Tim
>>>>
>>>>
>>>>>
>>>>> Jane
>>>>>
>>>>>
>>>>> Tim Quinn wrote:
>>>>>>
>>>>>> On Aug 16, 2010, at 7:49 PM, Byron Nevins wrote:
>>>>>>
>>>>>>> Has anyone looked at what the actual problem is?
>>>>>>
>>>>>> It's hard to do when QL passes on one's own system.
>>>>>>
>>>>>> And the relevant log files on Hudson that might shed some light -
>>>>>> the instance log files especially - disappear when the test
>>>>>> blithely deletes the instances.
>>>>>>
>>>>>> If someone has the same QL failure happening on his or her
>>>>>> system, please try running the QL tests step-wise:
>>>>>>
>>>>>> cd quicklook/cluster/clustersetup
>>>>>> ant all
>>>>>>
>>>>>> This is where the problems seem to occur.
>>>>>>
>>>>>> See if there are problems reported in the eein2 server log (if
>>>>>> there is one), or in the DAS server.log.
>>>>>>
>>>>>> - Tim
>>>>>>>
>>>>>>>
>>>>>>> On 8/16/2010 3:51 PM, Ken wrote:
>>>>>>>> I am building GF 3.1 rev 39757 with no changes. I am seeing 3
>>>>>>>> QL failures:
>>>>>>>>
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng] wsit_jaxws_tests
>>>>>>>> [testng] Tests run: 2, Failures: 0, Skips: 0
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng]
>>>>>>>> [testng] ######### FAILURE #########
>>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin
>>>>>>>> --interactive=false --echo=true --terse=true list-instances
>>>>>>>> --verbose=false --timeoutmsec 2000 --standaloneonly=false
>>>>>>>> --nostatus=false
>>>>>>>> [testng] eein1-with-a-very-very-very-long-name running
>>>>>>>> [testng] eein2 no response
>>>>>>>> [testng]
>>>>>>>> [testng]
>>>>>>>> [testng]
>>>>>>>> [testng] PASSED: createClusterTest
>>>>>>>> [testng] PASSED: createInstanceTest
>>>>>>>> [testng] FAILED: startInstanceTest
>>>>>>>> [testng] java.lang.AssertionError: Start instance failed ...
>>>>>>>> expected:<true> but was:<false>
>>>>>>>> [testng] at
>>>>>>>> test.clustersetup.ClusterSetupTest.startInstanceTest(ClusterSetupTest.java:93)
>>>>>>>>
>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>> [testng]
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng] cluster_setup
>>>>>>>> [testng] Tests run: 3, Failures: 1, Skips: 0
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng]
>>>>>>>> [testng] ######### FAILURE #########
>>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin
>>>>>>>> --interactive=false --echo=true --terse=true deploy
>>>>>>>> --force=false --precompilejsp=false --verify=false
>>>>>>>> --enabled=true --generatermistubs=false
>>>>>>>> --availabilityenabled=false --asyncreplication=true --target
>>>>>>>> eec1 --keepreposdir=false --keepfailedstubs=false
>>>>>>>> --logreportederrors=true --lbenabled=true
>>>>>>>> /space/ws/GlassFish/v3.1_unchanged/v3/tests/quicklook/dist/hellocluster/helloworld.war
>>>>>>>>
>>>>>>>> [testng] Application deployed successfully with name helloworld.
>>>>>>>> [testng] Command _deploy executed successfully on server
>>>>>>>> instance eein1-with-a-very-very-very-long-name
>>>>>>>> [testng] WARNING : Command _deploy did not complete
>>>>>>>> successfully on server instance eein2 : Authentication failed
>>>>>>>> for user: admin (Usually, this means invalid user name and/or
>>>>>>>> password)
>>>>>>>> [testng]
>>>>>>>> [testng]
>>>>>>>> [testng]
>>>>>>>> [testng] PASSED: clusterDeployTest
>>>>>>>> [testng] FAILED: clusterHelloWorldTest
>>>>>>>> [testng] java.lang.AssertionError: Accessing helloworld page
>>>>>>>> failed ... expected:<true> but was:<false>
>>>>>>>> [testng] at
>>>>>>>> test.hellocluster.ClusterHelloworldTest.clusterHelloWorldTest(ClusterHelloworldTest.java:76)
>>>>>>>>
>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>> [testng] SKIPPED: clusterUnDeployTest
>>>>>>>> [testng]
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng] cluster_helloworld
>>>>>>>> [testng] Tests run: 3, Failures: 1, Skips: 1
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng]
>>>>>>>> [testng] ######### FAILURE #########
>>>>>>>> [testng] asadmin --host localhost --port 4848
>>>>>>>> --interactive=false --echo=true --terse=true
>>>>>>>> delete-local-instance eein2
>>>>>>>> [testng]
>>>>>>>> [testng] The instance is running. Stop it and then re-run the
>>>>>>>> command.
>>>>>>>> [testng]
>>>>>>>> [testng] ######### FAILURE #########
>>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin
>>>>>>>> --interactive=false --echo=true --terse=true delete-cluster eec1
>>>>>>>> [testng]
>>>>>>>> [testng] remote failure: Exception while deleting the
>>>>>>>> configuration com.sun.enterprise.config.serverbeans.Cluster
>>>>>>>> :Cluster eec1 contains server instances eein2, and must not
>>>>>>>> contain any instances :
>>>>>>>> org.jvnet.hk2.config.TransactionFailure: Cluster eec1 contains
>>>>>>>> server instances eein2, and must not contain any instances
>>>>>>>> [testng] Cluster eec1 contains server instances eein2, and must
>>>>>>>> not contain any instances
>>>>>>>> [testng]
>>>>>>>> [testng]
>>>>>>>> [testng] PASSED: deleteInstanceTest
>>>>>>>> [testng] FAILED: deleteClusterTest
>>>>>>>> [testng] java.lang.AssertionError: Cluster unsetup failed ...
>>>>>>>> expected:<true> but was:<false>
>>>>>>>> [testng] at
>>>>>>>> test.clusterteardown.ClusterTeardownTest.deleteClusterTest(ClusterTeardownTest.java:66)
>>>>>>>>
>>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>>> [testng]
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng] cluster_teardown
>>>>>>>> [testng] Tests run: 2, Failures: 1, Skips: 0
>>>>>>>> [testng] ===============================================
>>>>>>>> [testng]
>>>>>>>>
>>>>>>>> Why are these tests failing? Is this a known problem?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Ken.
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Byron Nevins - Oracle Corporation
>>>>>>> Home: 650-359-1290
>>>>>>> Cell: 650-784-4123
>>>>>>> Sierra: 209-295-2188
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>>
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>