dev@glassfish.java.net

Re: Instance instance xxx is not reachable]

From: jothir ganesan <Jothir.Ganesan_at_Sun.COM>
Date: Wed, 27 Sep 2006 10:12:54 +0530

HI Nandini,

I have tried with older jdk versions also. Because the deployment works
fine if I do a normal deploy onto a cluster with jdk1.5.0_08 that is
bundled with appserver. Something else is causing the problem.

Sreeni,

Inspite of my instances and nodeagents running, I am unable to deploy.
This is blocking since build 15 as you know.
I'll try it on one machine setup as we discussed over phone today and
let you know the updates.

Thanks
Jothir

Sreenivas Munnangi wrote:

> Nandini Ektare wrote:
>
>> Sreenivas Munnangi wrote:
>>
>>> Sreenivas Munnangi wrote:
>>>
>>>> Nandini Ektare wrote:
>>>>
>>>>> Jothir
>>>>>
>>>>> Not sure if my following suggestion reached you. (I remember there
>>>>> was a thread of discussion and I assume Sreeni has already made
>>>>> you aware of various suggestions in that thread.)
>>>>> The only known environment change that happened between b13 and
>>>>> b15 was that of JDK minor version. So you might want to try with
>>>>> older version.
>>>>>
>>>>> "One more important thing... JDK build version changed between the
>>>>> two builds (switched to 1.5.0_08 on 8/29/06 so builds after that
>>>>> are using this b15, b16..)... I have not found any specific fixes
>>>>> related to java.io.ObjectOutputStream (see stack trace below)...
>>>>> but it will be good to check against the old JDK version to rule
>>>>> out this variable. JDK performance fixes had to be picked up...
>>>>> So this change could be a regression due to that.
>>>>>
>>>>> [#|2006-09-12T12:36:04.046+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise.system.tools.admin|_ThreadID=52;_ThreadName=event-handler-2;_RequestID=3ea01d2b-1c34-429c-b32d-d4d8cc24dbb3;|Instance:
>>>>> [instance3] not reachable
>>>>> com.sun.enterprise.admin.servermgmt.InstanceException:
>>>>> java.io.UTFDataFormatException
>>>>> at
>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.mangleInvokeException(InstanceProxy.java:150)
>>>>>
>>>>> at
>>>>> com.sun.enterprise.ee.admin.proxy.InstanceProxy.invoke(InstanceProxy.java:176)
>>>>>
>>>>> at $Proxy23.forwardEvent(Unknown Source)
>>>>> at
>>>>> com.sun.enterprise.ee.admin.event.EndPointHandler.run(EndPointHandler.java:88)
>>>>>
>>>>> at java.lang.Thread.run(Thread.java:595)
>>>>> Caused by: java.io.UTFDataFormatException
>>>>> at
>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1969)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream$BlockDataOutputStream.writeUTF(ObjectOutputStream.java:1812)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeUTF(ObjectOutputStream.java:816)
>>>>> at
>>>>> com.sun.enterprise.config.serverbeans.Domain.writeObject(Domain.java:606)
>>>>>
>>>>> at sun.reflect.GeneratedMethodAccessor145.invoke(Unknown Source)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>
>>>>> at java.lang.reflect.Method.invoke(Method.java:585)
>>>>> at
>>>>> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:917)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1339)
>>>>>
>>>>> at
>>>>> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1290)
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Nandini, I don't think the sole cause for "Instance instance1 is
>>>> not reachable" is java.io.UTFDataFormatException. Because we have
>>>> seen this error even without java.io.UTFDataFormatException. Seems
>>>> we need some way of reproducing this consistently to better
>>>> understand the issue.
>>>
>>>
>> So you are saying we have tried with earlier versions?
>
>
> No, I am not ruling out that possibility.
>
> It is possible that we have 2 issues here
> - one is java.io.UTFDataFormatException along with Instance instance1
> is not reachable error and the other is
> - Instance instance1 is not reachable error
>
> The error related to Instance instance1 is not reachable error was
> reported in the forum earlier
> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>
>> Reason to double check on this is that two of us reported that we
>> cannot reproduce the bug on b16 but both were using an older JDK
>> version. There were no changes related to this from admin and
>> deployment side(deployment team confirmed this) between these builds.
>> So in all probability it is to do with the setup/environment.
>>
>> About consistently reproducing this, I though Jothir reported
>> deployment always fails for a cluster.
>
>
> No, if we make sure that the instances are running and deploy then it
> works fine.
>
>>
>> Thanks
>> Nandini
>>
>>>>
>>> Also this seems an old issue
>>> http://forum.sun.com/jive/thread.jspa?forumID=114&threadID=64882
>>>
>>>>>
>>>>> Thanks
>>>>> Nandini
>>>>>
>>>>>
>>>>> jothir ganesan wrote:
>>>>>
>>>>>> Yes Kedar,
>>>>>>
>>>>>> I am stuck with this issue for 3 builds starting from build 16.
>>>>>> It is reproducible on Linux too. Please see bug#6467623.
>>>>>> Could you please explain what the EE QL is?
>>>>>>
>>>>>> Thanks
>>>>>> Jothir
>>>>>>
>>>>>> Kedar Mhaswade wrote:
>>>>>>
>>>>>>> Also, can you try the same thing on an older build? (b16, b17
>>>>>>> for instance)?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Kedar
>>>>>>>
>>>>>>> Kedar Mhaswade wrote:
>>>>>>>
>>>>>>>> Hi Jothir,
>>>>>>>>
>>>>>>>> I am seeing these errors while running the EE QL on Solaris.
>>>>>>>> Let us investigate ...
>>>>>>>> The tinderbox appears to be all green, however.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Kedar
>>>>>>>>
>>>>>>>> jothir ganesan wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> Could anyone please let me know the possible reasons for
>>>>>>>>> "EEADM0068: Instance instance xxx is not reachable" ? The
>>>>>>>>> exception shows up when trying to deploy an app on the cluster
>>>>>>>>> and the app fails to get deployed...
>>>>>>>>>
>>>>>>>>> I have the following setup:
>>>>>>>>>
>>>>>>>>> 3 nodeagents on 3 different machines with each nodeagent
>>>>>>>>> having one instance.
>>>>>>>>> Domain is started and the cluster,created on DAS, with the
>>>>>>>>> three instances - running
>>>>>>>>> Nodeagents- running
>>>>>>>>>
>>>>>>>>> Other configuration did:
>>>>>>>>> - hadb nodes running and ha-cluster is configured.
>>>>>>>>> - Access log buffer size ->
>>>>>>>>> cluster1-config.http-service.virtual-server.server.property.accessLogBufferSize
>>>>>>>>> set to 10 for cluster config
>>>>>>>>> - cluster1-config.http-service.keep-alive.timeout-in-seconds
>>>>>>>>> set to 0.
>>>>>>>>> - Some jdbc resources and connection pools created.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I have my nodeagents and instances up and running.
>>>>>>>>> Command start-cluster executed successfully.
>>>>>>>>> bash-3.00# asadmin list-instances --passwordfile passwordfile
>>>>>>>>> instance1 running
>>>>>>>>> instance2 running
>>>>>>>>> instance3 running
>>>>>>>>> Command list-instances executed successfully.
>>>>>>>>> bash-3.00# /space/91/appserver/bin/asadmin deploy --host
>>>>>>>>> localhost --port 4848 --user admin --passwordfile passwordfile
>>>>>>>>> --target cluster1
>>>>>>>>> /space/91/sift/agent/file-repository/70/Apps/SampleApplications/Clusterjsp1/clu
>>>>>>>>> sterjsp1.war
>>>>>>>>> Command deploy executed successfully.
>>>>>>>>> bash-3.00# asadmin list-node-agents --passwordfile passwordfile
>>>>>>>>> sqe-agent1 running
>>>>>>>>> sqe-agent2 running
>>>>>>>>> sqe-agent3 running
>>>>>>>>> Command list-node-agents executed successfully.
>>>>>>>>>
>>>>>>>>> But still when I try to deploy the app, I could see :
>>>>>>>>> [#|2006-09-26T16:03:20.722+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=198;_ThreadName=event-handler-0;instance1;_Request
>>>>>>>>> ID=4c5551da-d066-451e-a7a9-ee36c5a8ab47;|EEADM0068: Instance
>>>>>>>>> instance1 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> [#|2006-09-26T16:03:20.739+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=199;_ThreadName=event-handler-1;instance2;_Request
>>>>>>>>> ID=3c7c1df3-a47f-44a9-a418-e506bab400bf;|EEADM0068: Instance
>>>>>>>>> instance2 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> [#|2006-09-26T16:03:20.759+0530|WARNING|sun-appserver-ee9.1|javax.ee.enterprise
>>>>>>>>> .system.tools.admin|_ThreadID=200;_ThreadName=event-handler-2;instance3;_Request
>>>>>>>>> ID=da9f94fd-c35e-417d-ba3a-41c0cecc91c5;|EEADM0068: Instance
>>>>>>>>> instance3 is not reachable.|#]
>>>>>>>>>
>>>>>>>>> What else do I need to check?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> Jothir
>>>>>>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>