Tom and I exchanged a few e-mails off-line.
The admin command requests don't flow through the web container.
It seems plausible that there is some start-up issue that affects both
the AdminAdapter (which is why admin commands think the server is not
up) and the web container.
Is it possible that Grizzly adapters are started up asynchronously?
If so, then that could explain why an instance claims to have started
but an immediate asadmin command or web probe fails.
- Tim
On Aug 17, 2010, at 9:34 AM, Tom Mueller wrote:
> 13002 is a duplicate of 12967. It's easy to reproduce this problem
> by just doing the following:
> asadmin start-local-instance --node localhost ix; wget -S --no-
> proxy http://localhost:28080/
> This will show the web container returning a 500 error code
> immediately after the instance is started. This is being investigated.
> The reason that list-instances fails is because it calls __asadmin
> twice on the instance, so it is seeing the same problem.
>
> Tom
>
>
> On 8/17/2010 8:08 AM, Jane Young wrote:
>>
>> Tim Quinn wrote:
>>> Hi, Jane.
>>>
>>> On Aug 17, 2010, at 7:29 AM, Jane Young wrote:
>>>
>>>> Hi Tim,
>>>>
>>>> It's failing for me when running QL cluster_setup test by itself.
>>>> I think this is a performance regression. With the latest
>>>> changes, the startup time is taking longer. QL is doing "list-
>>>> instances" right after the "start-instance" command to check if
>>>> the instance is up and running. Since it's taking a longer time
>>>> for instance to startup, list-instances is returning false. I
>>>> added extra 5 sec wait in QL after start-instance and the tests
>>>> are passing. Same goes for cluster_helloworld tests. I'll
>>>> commit the changes in QL tests.
>>>
>>> If start-instance reports success before the instance is really
>>> ready to do work, then that's a bug. Inserting the 5-second pause
>>> makes sense to get the QL tests passing again but I hope someone
>>> will investigate why the instance claims to be up when it is not
>>> really up.
>> Agreed that this is a workaround in QL. Bug filed for the start-
>> instance command: https://glassfish.dev.java.net/issues/show_bug.cgi?id=13002
>> What about the cluster_helloworld test - is that also a bug? QL is
>> trying to access the helloworld app right after deployment and it's
>> taking a longer time to generate an app.
>>
>>
>>>
>>> - Tim
>>>
>>>
>>>>
>>>> Jane
>>>>
>>>>
>>>> Tim Quinn wrote:
>>>>>
>>>>> On Aug 16, 2010, at 7:49 PM, Byron Nevins wrote:
>>>>>
>>>>>> Has anyone looked at what the actual problem is?
>>>>>
>>>>> It's hard to do when QL passes on one's own system.
>>>>>
>>>>> And the relevant log files on Hudson that might shed some light
>>>>> - the instance log files especially - disappear when the test
>>>>> blithely deletes the instances.
>>>>>
>>>>> If someone has the same QL failure happening on his or her
>>>>> system, please try running the QL tests step-wise:
>>>>>
>>>>> cd quicklook/cluster/clustersetup
>>>>> ant all
>>>>>
>>>>> This is where the problems seem to occur.
>>>>>
>>>>> See if there are problems reported in the eein2 server log (if
>>>>> there is one), or in the DAS server.log.
>>>>>
>>>>> - Tim
>>>>>>
>>>>>>
>>>>>> On 8/16/2010 3:51 PM, Ken wrote:
>>>>>>> I am building GF 3.1 rev 39757 with no changes. I am seeing 3
>>>>>>> QL failures:
>>>>>>>
>>>>>>> [testng] ===============================================
>>>>>>> [testng] wsit_jaxws_tests
>>>>>>> [testng] Tests run: 2, Failures: 0, Skips: 0
>>>>>>> [testng] ===============================================
>>>>>>> [testng]
>>>>>>> [testng] ######### FAILURE #########
>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin --
>>>>>>> interactive=false --echo=true --terse=true list-instances --
>>>>>>> verbose=false --timeoutmsec 2000 --standaloneonly=false --
>>>>>>> nostatus=false
>>>>>>> [testng] eein1-with-a-very-very-very-long-name running
>>>>>>> [testng] eein2 no response
>>>>>>> [testng]
>>>>>>> [testng]
>>>>>>> [testng]
>>>>>>> [testng] PASSED: createClusterTest
>>>>>>> [testng] PASSED: createInstanceTest
>>>>>>> [testng] FAILED: startInstanceTest
>>>>>>> [testng] java.lang.AssertionError: Start instance failed ...
>>>>>>> expected:<true> but was:<false>
>>>>>>> [testng] at
>>>>>>> test
>>>>>>> .clustersetup
>>>>>>> .ClusterSetupTest.startInstanceTest(ClusterSetupTest.java:93)
>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>> [testng]
>>>>>>> [testng] ===============================================
>>>>>>> [testng] cluster_setup
>>>>>>> [testng] Tests run: 3, Failures: 1, Skips: 0
>>>>>>> [testng] ===============================================
>>>>>>> [testng]
>>>>>>> [testng] ######### FAILURE #########
>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin --
>>>>>>> interactive=false --echo=true --terse=true deploy --
>>>>>>> force=false --precompilejsp=false --verify=false --
>>>>>>> enabled=true --generatermistubs=false --
>>>>>>> availabilityenabled=false --asyncreplication=true --target
>>>>>>> eec1 --keepreposdir=false --keepfailedstubs=false --
>>>>>>> logreportederrors=true --lbenabled=true /space/ws/GlassFish/
>>>>>>> v3.1_unchanged/v3/tests/quicklook/dist/hellocluster/
>>>>>>> helloworld.war
>>>>>>> [testng] Application deployed successfully with name helloworld.
>>>>>>> [testng] Command _deploy executed successfully on server
>>>>>>> instance eein1-with-a-very-very-very-long-name
>>>>>>> [testng] WARNING : Command _deploy did not complete
>>>>>>> successfully on server instance eein2 : Authentication failed
>>>>>>> for user: admin (Usually, this means invalid user name and/or
>>>>>>> password)
>>>>>>> [testng]
>>>>>>> [testng]
>>>>>>> [testng]
>>>>>>> [testng] PASSED: clusterDeployTest
>>>>>>> [testng] FAILED: clusterHelloWorldTest
>>>>>>> [testng] java.lang.AssertionError: Accessing helloworld page
>>>>>>> failed ... expected:<true> but was:<false>
>>>>>>> [testng] at
>>>>>>> test
>>>>>>> .hellocluster
>>>>>>> .ClusterHelloworldTest
>>>>>>> .clusterHelloWorldTest(ClusterHelloworldTest.java:76)
>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>> [testng] SKIPPED: clusterUnDeployTest
>>>>>>> [testng]
>>>>>>> [testng] ===============================================
>>>>>>> [testng] cluster_helloworld
>>>>>>> [testng] Tests run: 3, Failures: 1, Skips: 1
>>>>>>> [testng] ===============================================
>>>>>>> [testng]
>>>>>>> [testng] ######### FAILURE #########
>>>>>>> [testng] asadmin --host localhost --port 4848 --
>>>>>>> interactive=false --echo=true --terse=true delete-local-
>>>>>>> instance eein2
>>>>>>> [testng]
>>>>>>> [testng] The instance is running. Stop it and then re-run the
>>>>>>> command.
>>>>>>> [testng]
>>>>>>> [testng] ######### FAILURE #########
>>>>>>> [testng] asadmin --host localhost --port 4848 --user admin --
>>>>>>> interactive=false --echo=true --terse=true delete-cluster eec1
>>>>>>> [testng]
>>>>>>> [testng] remote failure: Exception while deleting the
>>>>>>> configuration
>>>>>>> com.sun.enterprise.config.serverbeans.Cluster :Cluster eec1
>>>>>>> contains server instances eein2, and must not contain any
>>>>>>> instances : org.jvnet.hk2.config.TransactionFailure: Cluster
>>>>>>> eec1 contains server instances eein2, and must not contain any
>>>>>>> instances
>>>>>>> [testng] Cluster eec1 contains server instances eein2, and
>>>>>>> must not contain any instances
>>>>>>> [testng]
>>>>>>> [testng]
>>>>>>> [testng] PASSED: deleteInstanceTest
>>>>>>> [testng] FAILED: deleteClusterTest
>>>>>>> [testng] java.lang.AssertionError: Cluster unsetup failed ...
>>>>>>> expected:<true> but was:<false>
>>>>>>> [testng] at
>>>>>>> test
>>>>>>> .clusterteardown
>>>>>>> .ClusterTeardownTest
>>>>>>> .deleteClusterTest(ClusterTeardownTest.java:66)
>>>>>>> [testng] ... Removed 26 stack frames
>>>>>>> [testng]
>>>>>>> [testng] ===============================================
>>>>>>> [testng] cluster_teardown
>>>>>>> [testng] Tests run: 2, Failures: 1, Skips: 0
>>>>>>> [testng] ===============================================
>>>>>>> [testng]
>>>>>>>
>>>>>>> Why are these tests failing? Is this a known problem?
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Ken.
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Byron Nevins - Oracle Corporation
>>>>>> Home: 650-359-1290
>>>>>> Cell: 650-784-4123
>>>>>> Sierra: 209-295-2188
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>